devopsy.kinsta.cloud

Tag: automation

  • Tanium Uses SBOMs to Automate Vulnerability Remediation

    Tanium Uses SBOMs to Automate Vulnerability Remediation

    Tanium this week added the ability to detect libraries and software packages with known vulnerabilities within a software bill of materials (SBOM) manifest that can then be used to automate remediation of endpoints running vulnerable code.

    Pete Constantine, senior vice president of product management for Tanium, said the Tanium Software Bill of Materials (SBOM) module provides IT teams with both the ability to discover where vulnerable code is running in real-time and automatically apply any required software patch.

    The Tanium SBOM tool examines the contents of individual files wherever they reside in IT environments to make it simpler to immediately discover where, for example, vulnerable instances of the Log4j log management software or OpenSSL software for securing communications are running. Today many IT teams are spending weeks trying to track down every potential instance of code each time a new vulnerability is discovered.

    Once those vulnerabilities are discovered, IT teams can then opt to use a Tanium Patch tool to remediate vulnerabilities or use other Tanium management tools to kill specific processes or uninstall applications, noted Constantine.

    Awareness of the need for SBOMs to secure software supply chains has increased sharply in the wake of a series of high-profile cybersecurity breaches. However, few organizations have the tools in place to operationalize the data collected in SBOMs that are essentially little more than a list of components employed to build an application. The goal is to enable IT teams to decide whether to go ahead and deploy an application based on the level of threat the vulnerabilities included represent.

    It’s unlikely that vulnerabilities will ever be completely eliminated from applications running in production environments, but it’s clear there is a need to be able to quickly identify issues based on the level of severity any vulnerability represents. The trouble is that an SBOM is only the first step toward achieving the level of remediation automation that is required to address vulnerabilities discovered in applications after they have been deployed.

    Hopefully, all the focus on securing software supply chains will lead to a narrowing of the divide that has historically existed between cybersecurity and application development teams. Developers still need to create patches to fix issues, but overall the number of vulnerabilities that find their way into production environments should decline as more organizations embrace DevSecOps best practices before applications are deployed.

    In the meantime, however, given how much developers today reuse code, the odds are good multiple vulnerabilities have already been widely propagated across an application environment. In fact, the amount of technical security debt that organizations face today is simply staggering when considering the total number of potential vulnerabilities that need to be remediated. In the absence of IT automation capabilities, it might take the better part of a decade to resolve all those vulnerabilities.

    On the plus side, however, the tools and platforms required to automate remediations are now integrated with SBOMs that, at the very least, make pinpointing where vulnerabilities are located easier. The challenge now is defining a set of end-to-end processes that make fixing all those vulnerabilities a much less arduous task for all concerned.

  • Red Hat and IBM Pledge to Bring NLP to Ansible

    Red Hat and IBM Pledge to Bring NLP to Ansible

    At the Ansiblefest 2022 conference today, Red Hat and its parent company IBM announced the Project Wisdom initiative to bring natural language processing (NLP) to the open source Ansible IT operations automation platform.

    At the same time, Red Hat announced that the curated instances of Ansible from Red Hat are now available in the marketplaces operated by Amazon Web Services (AWS) and Microsoft.

    Richard Henshall, senior manager of Ansible product management at Red Hat, said the goal is to make IT automation much more accessible to a broader range of end users and IT professionals that lack the declarative programming expertise to use YAML files to automate IT processes. Instead, Red Hat and IBM Research teams are pledging to combine their expertise to create an artificial intelligence (AI) model that would enable anyone to use natural language to automate an IT process.

    Ansible has emerged as the de facto open source standard for automating a wide range of IT processes, but the number of processes that could be automated still far exceeds the number of IT processes that are currently automated. In recent years, IT automation tools have become more accessible thanks in part to the rise of low-code and no-code tools, but some level of programming expertise is still required. Project Wisdom is being launched to eliminate the need to acquire those skills before being able to automate a process.

    Of course, there will still be complex processes that may require the ability to programmatically manipulate YAML files, but the number of trivial IT processes that could be automated is still incalculable. In theory, a natural language interface for Ansible would greatly reduce the time and effort required for anyone to automate an IT process regardless of their skill level.

    Interest in IT automation is rising once again as the global economy continues to soften. Organizations are looking to automate as many processes as possible in their efforts to keep IT staff size as small as possible. The more time IT teams spend on manual processes the less they will be able to manage IT environments that get more complex with each passing day.

    It’s not clear to what degree IT processes can be automated, but as businesses become more dependent on IT, the line between IT automation and business processes continues to blur. Natural language processing capabilities could make it simpler for organizations to greatly accelerate digital business transformation initiatives.

    Regardless of the level of IT automation achieved, it’s clear the way IT is managed is about to evolve once again. The days when IT teams configured and provisioned IT resources using fragile custom scripts are coming to an end as platforms like Ansible are more widely adopted. The only issue that remains to be seen is how quickly IT processes can be automated and whether the rate of automation can keep up with the volume of applications currently being developed.

  • 5 Ways to Transform DataOps With Human-in-the-Loop Automation

    5 Ways to Transform DataOps With Human-in-the-Loop Automation

    We are in the middle of a data renaissance. Today, it’s not just about data instrumentation but also learning how to make DataOps a real business advantage for the entire organization. Data has become an inextricable part of products, used to enhance the quality of user experiences. Reliable access to data and the integrity of that data is imperative to drive innovation and business success.

    But just like any process, problems will arise. There will be pipelines that break or data that isn’t instrumented properly. The question then becomes, how can teams more quickly identify problems and continually improve? How can the journey toward data integrity become one of iteration and improvement?

    DataOps teams are by no means strangers to automation—automated CI/CD pipelines and infrastructure-as-code (IaC) are a huge part of their skill set. But there are some components of automation that are less pervasive throughout DataOps, ones that can help promote data integrity and encourage a culture of improvement. 

    It all starts with a different approach to automation—a human-in-the-loop (HITL) approach. Human-in-the-loop automation enables parts of a process to be fully automated while enabling humans to step in at critical points to take action, make decisions and decide the path forward.

    Here are five ways to use human-in-the-loop automation to drive continuous improvement and achieve optimal data integrity.

    Create Better, Earlier Interruption of Problems

    As is life, inevitably things break, issues arise and teams need to step in to remediate. The goal should be to enable better interruption of these problems when they do occur—identifying issues earlier and bringing the right stakeholders together more easily and swiftly.

    HITL automation can help teams not just identify issues earlier but kickstart the incident response and coordinate across teams and stakeholders. Ideally, automated workflows can be triggered from an incoming signal; for instance, from an observability tool like DataDog or BigPanda. An incident response could be started automatically, triggering further automation—like creating tickets, starting a Zoom meeting, creating a Slack channel and bringing together the right team members. 

    By instantly bringing the right people together, the team can investigate collaboratively and more quickly understand the state of the issue. HITL automation should then enable teams to run further scripts to take action and remediate as needed.

    Bake Improvement Into the Process Through Self-Documenting Workflow

    Many companies today are asking, “How do we make sure that we have that full integrated loop, down to the collection point?” Communication between teams is critical to this endeavor. But even before teams can collaborate on improvements, they need to understand the current state of their data so that if, for instance, a team finds that a report isn’t providing what’s needed, there’s a really clean way to get that communication back to the DataOps team to improve the instrumentation in the first place.

    HITL automation can bake improvement into DataOps workflows by automatically documenting every human and machine action throughout a process. Because it’s connected through APIs to all a team’s tools and services, a self-documenting workflow slurps up every action and creates an audit trail. This audit trail is the basis by which teams can more accurately understand not just the integrity of their data, but also how teams are implementing and running processes.

    Bring Human Data to the Forefront of Improvement

    DataOps teams play a big role in making sure the right data is being collected. However, a lot of people are still using systems that are very manual. They may have a good understanding of their log and monitoring data but still don’t have a good sense of the human data around their organization—what processes and workflows people are running.

    Understanding how humans run processes is just as important as understanding technology problems. The self-documenting workflow described above should include documentation of human data, from manually-run scripts to Slack or Teams conversations and, ideally, even auto-update tickets so even maintaining a system of record becomes less manual. Only by looking at the full picture—of human and machine data together—can teams find areas of improvement in their automation, communication and workflows.

    Take an Incremental Approach

    Don’t let “perfect” be the enemy of “done”. Every great transformation I’ve seen started with a small team, a great mandate, and strong support from the executive team. Adding tons of new technology and hiring more people may not be the silver bullet teams need.

    Instead, teams should look at the expertise they have in-house first—the people, technology and processes. Then, begin by taking an incremental approach. HITL promotes incremental, approachable automation by enabling teams to first codify institutional knowledge, analyze, learn and then begin to automate pieces in small batches.

    Diving head-first into a fully automated approach can leave teams in a worse state than nothing at all. Automation is a journey—we learn by doing, by recording human and machine data and then, using insights as our guide, begin to find real value in small doses of automation. These small steps can make a huge impact and, over time, add up to substantial business value.

    Buy for Industry Standards and Build for the Gaps

    As I mentioned above, teams should first start by evaluating their current resources. What skill sets and technology do they already have and how can they use those resources to achieve their data goals? At the same time, many teams starting their automation journey face challenges—lack of development resources, scripts that live on one person’s machine, difficulty bringing all the human and machine data together into a recorded timeline.

    To help data teams effectively and reliably implement HITL automation, investment into an off-the-shelf platform can help bridge the gaps teams and organizations have. Platforms should enable automation through low-code interfaces while providing code-level customization. This combination enables teams to buy for industry standards while building for the gaps—not managing the platform means development resources can be focused on expanding and improving automation.

    Platforms should enable teams to easily connect to APIs and adapt to custom APIs. The continuous change of DevOps processes and tooling coexist with rapidly changing APIs. Therefore, the most promising automation platforms must manage the complexity of APIs, enabling users to focus on the intent, not the mechanics of custom APIs. Automation technologies should seamlessly handle identity, authentication, pagination and caching. 

    Find Leverage With HITL Automation and Drive Business Impact

    As data becomes increasingly ingrained in product life cycles and user experience, automation holds the key to faster innovation, product quality and enhanced reliability. Creating a culture of continuous improvement and implementing technology like HITL automation that promotes data integrity throughout the product life cycle is where I’ve seen the greatest leverage for organizations. 

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2022 – October 24-28, 2022.

  • DevOps and Hybrid Cloud: Life in the Fast Lane, Part Two

    DevOps and Hybrid Cloud: Life in the Fast Lane, Part Two

    In the early years of cloud computing, cost savings were the major driver of adoption. These days, however, organizations are increasingly faced with spiraling expenditures, waste and a lack of control. As cloud usage bills and inefficiencies start to balloon, these challenges are beginning to undermine the benefits of migrating in the first place.

    The truth is that optimizing costs in the cloud is more taxing than many organizations first expect. Identifying cost savings that can be made isn’t generally the hard part; effectively implementing those savings is where the real challenge lies.

    A lack of control over infrastructure means that many organizations are failing to make the changes needed to truly realize these gains. So common are these challenges that there is even a word for the process to deal with them: FinOps. FinOps, at its core, is the culture and practice of creating visibility and accountability to manage cloud penetration throughout an organization, and it’s an area that is growing rapidly.

    This is the second article in my series that looks at what needs to change so that organizations can really start embracing the opportunities provided by hybrid cloud and DevOps. In this installment, I’m going to take a look at the areas of FinOps, cloud waste and automation. In doing so, I hope to show that there is indeed life in the hybrid cloud and DevOps fast lane if handled the right way.

    Get Ready for FinOps

    If you think hybrid cloud and the topics of FinOps and, more specifically, cloud waste, haven’t come up during discussions, you’ve probably been living under a rock.

    It naturally follows that the more agile an organization becomes the harder it needs to work to keep track of waste. Exponential growth is great, but without the right controls a cloud budget will disappear before there has been a chance to see any returns; there are even examples of companies migrating and then deciding to roll back—such are the concerns around lack of control and rising costs.

    Based on survey responses from over 750 cloud decision-makers, Flexera found earlier this year in its 2022 State of the Cloud Report that organizations continue to waste significant amounts of money in the cloud. A full 66% of respondents said that cloud usage is “higher than initially planned this year”, and they estimated that they are wasting 32% of their overall cloud spend. A good five years into full cloud migration and most organizations still haven’t got a handle on costs.

    The tricky part is that waste can rear its head at many stages over the course of a project, which is why it’s so important to maintain control from day one.

    Establishing a baseline of an environment that is deemed to be the known state means that what is versioned can become a source of trust and the desired state of an organization’s cloud infrastructure. How is this possible? Simple: By creating a culture and practice of visibility and accountability throughout an organization. 

    How do you do that? Yes, you guessed it: FinOps.

    Tackling Cloud Waste Head-On

    The ideal time and place to start establishing robust FinOps policies and reducing waste come when the move has been made to start embracing a dual DevOps/hybrid cloud approach. The good news is that the principles of FinOps are easy to follow and can be applied at various stages of an application management life cycle and DevOps and hybrid cloud journey.

    First, teams need to estimate financial and environmental costs ahead of deployment, something which can be particularly challenging when infrastructure-as-code (IaC) tools such as Terraform, Ansible or Puppet are being used, but which is essential to successful FinOps. Once a team has decided which cloud provider they will be using, how much they plan to spend and, more generally, what criteria will be governing these decisions, effective tag management comes in.

    Tagging is the basis for achieving a structured cloud infrastructure, especially in multi-cloud environments that feature different resources being used for different purposes. Tagging means that users can add descriptive metadata (tags) to cloud infrastructure to identify key resources or values such as staging, development or production.

    In a rapidly growing cloud environment, consistent and comprehensive tag management helps to keep track of what is happening in terms of cost, usage, availability, performance and security across a constantly transforming cloud infrastructure. Once a project has been deployed, these tags need to be managed and maintained across all cloud providers, something which is critical for the management of waste, but is often a huge and messy task.

    Good tags help drive good decision-making and the more options there are for categorizing data, the better. To truly get a handle on tag management, finding a platform that can enable users to visualize and study multi-cloud costs in a central place makes it possible to establish that all-important baseline and start to reverse engineer invoices, optimize monthly costs and tackle waste head-on.

    One Cloud, One Strategy

    There was a time a few years ago that we used to hear people say this a lot: “I’ll migrate all my assets to a single cloud provider.” Cloud providers were pushing migration to a one-cloud strategy; considered completely normal at the time, but which has since been shown to be flawed, especially in the world of enterprise.

    Segregation of responsibility is perhaps the best example of why—there are simply too many responsibilities that keep a modern business afloat to risk putting all your eggs in one (cloud) basket. Consider on-premises legacy apps, sensitive data, critical workloads, the production life cycle, ongoing applications without too many peaks and even backend apps. Then consider the cloud providers, new cloud-first applications, public data, non-critical workloads, the development life cycle and frontend applications dealing with high traffic.

    Now imagine all of that suddenly disappears.

    Hybrid cloud means eggs in multiple baskets. Yes, it adds a layer of complexity, but when so much is at stake, it’s worth it. And when you have defined where you want to host everything for each application, it becomes much easier to gain clarity. The challenge is that all cloud providers have their own ecosystem, which will either be private or public.

    These ecosystems make it possible for everything to be integrated, but what is often not immediately obvious is that even though they are integrated, they are not always aligned. It’s in the interests of the cloud provider to provide the best tools to use their platform but not necessarily the best user or developer experience which is something that requires specialist expertise. That usually leads to a certain degree of lock-in as you generally cannot use their tools for managing other applications that aren’t part of the infrastructure.

    This lack of transparency and visibility into other cloud providers can get in the way and impact the ability to get the most out of a dual DevOps and hybrid cloud strategy. However, by adopting a hybrid cloud approach, teams can choose the best cloud provider for each application, putting the team in the best position possible to provide the best services possible.

    And finally, Always Consider the DevX Factor

    For DevOps to thrive, it is important for companies to stay independent both for their own sovereignty and for creating the best developer experience, or DevX (a topic I have explored in a previous article for devopsy.kinsta.cloud).

    It’s easy to fall into the trap of using all the automation capabilities available from each cloud provider, but that locks an organization into a situation where it needs deep expertise across a wide variety of technologies. Adopting a one-cloud, one-automation strategy means that it would never be possible to migrate automation or reuse it in different application deployment processes across various clouds.

    This is where automation of configuration management, IaC, CI/CD pipelines and the DevOps and hybrid cloud platform has a significant role to play. Automation on an organization’s own terms is arguably the most critical element in the race to reduce time to market and improve quality where concepts such as shift left are adding complexity and increasing the responsibilities piled on top of developers.

    Using automation technologies to maintain autonomy and favor cross-subject expertise and team mobility gives organizations the freedom and flexibility to save time, money and roll out better DevX. Ultimately, it also helps to create the best possible environment for organizations to truly start enjoying a DevOps and hybrid cloud life in the fast lane.

  • Testing Automation has Become a Necessity – Techstrong TV

    Testing Automation has Become a Necessity – Techstrong TV

    Guljeet Nagpaul, Chief Product Officer for ACCELQ, explains why automating testing has become a necessity. The video is below followed by a transcript of the conversation.

    Mike Vizard: Hey, guys. Thanks for the throw. We’re here with Guljeet Nagpaul, who is chief product officer for ACCELQ, and we’re talking about all things related to testing. Guljeet, welcome to the show.

    Guljeet Nagpaul: Thanks, Mike. Thanks for having me.

    Mike Vizard: Do you think the whole testing focus has increased somewhat in the last few months with the whole shift to digital transformation, and people are trying to develop more applications than ever? I think it’s kind of critical to get it right the first time these days. I think the tolerance for experimenting on end users, especially in a digital business context, is a lot lower, but is there some sort of fundamental shift going on with testing that you’re seeing?

    Guljeet Nagpaul: Yeah, both things you mentioned, Mike, are right on. The digital transformation has clearly been on steroids almost now for the past couple of years, since we’ve seen the world kind of changing around us with remote work. A lot of processes which were even in healthcare and financial space, which were kind of buried with red tape, all of them have been kind of almost on steroids to become more automated, and organizations have had to kind of put their transformation objectives much more on a fast track. That, like you rightly said, automatically leads to having to ship the applications must faster to their customers to run their businesses faster. The need for automated testing for shipping their software faster gets kind of even more critical.

    Mike Vizard: Where is that automation taking place? Are we automating tests for the developer as we shift things left, or is the automation more in the CI/CD DevOps platform, and we’re trying to automate it there because that may be the place where the application itself comes together; otherwise, each developer is testing a small piece of something but maybe not the whole thing and then?

    Guljeet Nagpaul: That’s exactly right. We live in a world where there is an app for everything. That has been true from a consumer perspective, but now, interestingly, that is more so true on our enterprise application side. Back in the day, if you remember, we had a couple of ERPs and then we had some homegrown systems, and that’s kind of how the IT landscape of organizations looked. But now, with the onset of cloud apps, with packaged applications, with vertical applications, there is an app for everything for enterprises to be able to go more automated. What that translates to is a highly-integrated enterprise application ecosystem. What that means is unit testing or just shifting left in terms of automation is not going to be enough.

    You would really need that end-to-end validation, and each app has at least five upstream and five downstream integrations, which means the real automation, and the real customer experience in terms of your end-to-end workflow has to be automated across the board. That DevOps CI/CD pipeline you talked about, that gets even more kind of focused in terms of being able to validate that end-to-end workflow. That is kind of where we are really seeing the need for automation, more than kind of what we would have thought from a shift-left perspective. That’s absolutely necessary because you want to build a stronger product. You don’t want to be reactive in building a strong product, you want to bake quality into the product, but then downstream, you want to make sure your end-to-end workflows work as designed in terms of an integrated end-to-end system.

    Mike Vizard: How far down the path are we to achieving that goal, because I think somewhere along the line everybody maybe thinks that testing is somebody else’s job, but what’s the maturity level of testing these days in terms of automation?

    Guljeet Nagpaul: Yeah, like with a few different functions within our IT, we kind of see kind of a full circle come back, right? A lot of testing was starting to get perceived as commoditized, and we saw a lot of testing kind of getting shifted from that perspective as a cost function. But, quickly, organizations realized that without continuous testing, there is no continuous delivery, and that’s really the crux of it, that if you can’t have a dependable, reliable, continuous test automation to bridge that gap between your dev and ops, there is no DevOps, because that’s the key bridge between your dev and ops, right? With that notion, the testing maturity level has struggled.

    To be honest, a lot of automation approaches, and I spent over a decade with HP, a lot of traditional automation tools and approaches have focused more on silos because our app ecosystem was not that integrated back in the day, right? It used to kind of have integration, but not at the level that you’re seeing today, so the need for an automation approach that can span across these technology stacks is now kind of more new and is now more critical. The testing maturity in terms of what approaches are out there has struggled, but we are now starting to see customers really needing that across-technology-stack approach, and that’s kind of where ACCELQ has a sweet spot.

    Mike Vizard: Do you think the rise of micro-services is forcing the automated testing issue because we’re reaching a level of complexity that can’t really manually test all these things?

    Guljeet Nagpaul: I would say yes. Just in the last five years, interestingly, Mike, we have seen API testing as part of critical proof-of-concepts to when customers come and try a platform like ACCELQ out in terms of seeing whether it’s fit for their needs or not. Just in the last five years, we have seen at least three times the interest in making sure that APIs can be validated apart from their UI. But, like I said, it’s really more about how that validation can be done across the technology stacks. To give you an example, a healthcare customer of ours, one of the top five companies in the world, they had a use case where a patient walks into a pharmacy to put in their prescription, but now with COVID and everything, the automation is kind of boosted.

    Now they can key in their prescriptions directly through their mobile apps connected to their healthcare providers. Once that prescription is keyed in, there are at least 20 downstream processes that are kicked in, including mainframe systems, micro-services, a database that’s maybe sitting on a cloud, but then coming back to a pharmacy system which is still living in an old-school desktop world. The real demand, what we are seeing is kind of this across technology side, because that’s what the user experience is.

    Mike Vizard: How smart can testing get? We hear a lot about artificial intelligence these days. Will the whole thing become automated or what will be the role of the tester, per se?

    Guljeet Nagpaul: Yeah, that’s a great question. AI unfortunately gets thrown around very loosely these days, especially more so with talking about how AI is the platform. We have looked at it always as a tool in our toolkit. Ever since the inception of our platform, we have always looked at how we can leverage AI to make things more efficient as opposed to saying AI is the platform. We have always looked at it more realistically. If you forget AI for a second, that’s kind of how as people we have embraced technology. In various aspects, technology has enabled us to increase our throughput regardless of which process area are we picking, and that is kind of how we see AI. If it’s leveraged accurately, it’s going to increase the throughput of our test engineers. It is going to increase the throughput of our automation engineers. At various process areas that then test automation life cycle, that’s kind of how we’ve looked to leverage AI.

    Mike Vizard: Do you think we’re approaching a point where the number of applications being built is overwhelming the testers that are out there, and the pace of that work is getting to a point where they become a bottleneck?

    Guljeet Nagpaul: Yeah, I was having a discussion on the same lines with our Forrester analysts, and one of the interesting discussion points around that was we all as testers within the DevOps space have to almost take a step back and understand what needs to be tested before going about kind of in the traditional manner testing it. Yes, it can be overwhelming with so many apps coming together so quickly in terms of being able to serve our businesses, but a lot of these apps which are getting shipped for our businesses, for these organizations are cloud apps, or maybe packaged apps, vertical apps. All they need is configuration and integration with other apps, and then they can kind of start doing the job in terms of the outcome that the businesses are expecting. It almost needs some level of refactoring, a new mindset when you think about in this new world of facts, how should you be testing and what you should be testing to get the maximum bang for your buck, so it’s easy to get overwhelmed. But, with automation and the right approach to what needs to be tested, you can kind of be more efficient with your efforts.

    Mike Vizard: Who writes the tests these days? It seems to me a lot of time and effort has been spent on creating the tests, and, as we automate that process, can we get to the point where the machines can create the tests? I mean, to what degree do I need to create the tests versus run the tests?

    Guljeet Nagpaul: Yeah, that’s an interesting point. The way we look at it is, unfortunately, the way testing and automation specifically has evolved, the traditional approaches I talked about, they have lived on two ends of the spectrum, Mike, one end of the spectrum being very record-and-playback oriented. But, we have seen in the past 20 years that automation is not new; it has been around for at least a couple decades. Back when we released Windrunner, and I don’t know if that rings a bell, I was part of the original Windrunner team, at one point we had about 70 percent market share, and record-and-playback was the thing automation. But, in the last ten years, there have been a plethora of tools which have come on the same lines of record-and-playback but we have seen that they just are not scalable enough to deal with the changing technology stack. They are just not real-world enough to work across technology stacks, and a lot of other things in terms of design challenges and so on.

    The other end of the spectrum that we live in is open source, and you kind of pointed to how testing had been shipped out as a cost function, and open source made a big kind of wave into the test automation area, but that became a specialist function. The reason this is important, to your point of the question, is when automation has become a specialist function, because we have seen about 60 to 65 percent market share ‒ Gartner did a survey of open source taking over a lot of kind of testing and automation approaches. But, when you see that, basically it becomes a specialized function, and all my testing team is a mixed bag of testers, maybe some automation engineers, but a lot of business subject matter expertise. If you really want a strong test approach, you need to make the subject matter experts people who understand business rules.

    That needs to be validated as part of the automation, and that’s kind of where both ends of the spectrum have to meet, where a non-technical, non-specialized tester should also be able to be part of that automation. That can only become if our platforms are smarter, right? Maybe depending on that suite to be adaptable, maybe bridging machine in certain cases, to your point, to be able to write smarter test cases, to basically have your automation more leaner and meaner so that it can do more with kind of less in terms of authoring and design changes and change with so many releases happening, so absolutely. I mean, testing has to get out of the specialized function role.

    Mike Vizard: All right. We hear a lot of focus these days on security, of course. Do you think that security ultimately will become part of a quality assurance testing process, because it’s not clear to me that those are two distinct functions anymore. They might be joined at the hip in some fashion.

    Guljeet Nagpaul: I sure hope so. The strongest of the software and the organizations just in the past one year, we have seen outages which we would have never imagined, right? AWS went out a few times, Facebook, Meta, went out a few times, and you name it. Security has to become an integral part, and we are seeing this more and more. As part of functional regression tests, customers want to have those security regression tests. Security lived in two worlds until now. One was static code analysis, which is, “Let’s do static code analysis,” which is good, it’s kind of like shifting left to your unit tests. Yes, you’ve got to find your vulnerabilities right when your code is written, but, once the code is in action, what vulnerabilities come can be a different game.

    Static code analysis is where security lived, and, like you said, that’s why it was a little bit in the different world. Then, you had dynamic security firewalls, which are kind of more glorified firewalls. But, in the middle, that functional testing, leveraging security validation is a gap in the industry right now, and we are certainly looking to fill that gap. We have already helped a lot of Fortune 100 customers, including certain financial services customers, fill in that gap, leveraging the functional automation approach because they’ve had some solutions around these two things but functional is where it needs to get seated into, right? Like you said, it has to become mainstream into the testing and function automation space.

    Mike Vizard: All right. Other than hanging out on your website, what is your best advice to folks about how to get to this continuous testing nirvana, I guess, because, ultimately, time is money, so every time an application has a disjoined experience, somebody is losing money somewhere, right?

    Guljeet Nagpaul: Totally. Automation first. Automation first is the key. A lot of traditional mindset still remains at, “Let’s write my manual tests and let’s write the manual tests them into convert them into automation.” The mindset has to change. The organization approach has to change to automation first. For automation first, you can only do automation first if it’s not a specialist function. Don’t get caught up in getting buried into a technical debt. You don’t want a million lines of code to test a million lines of code, right? Figure a more robust solution and go automation first, and that will set you on the right path.

    Mike Vizard: All right. Guljeet, thanks for being on the show.

    Guljeet Nagpaul: Thanks, Mike. Thanks for having me.

    Mike Vizard: All right. Back to you guys in the studio.

  • How IaC Helps Relieve Development Pain Points

    How IaC Helps Relieve Development Pain Points

    Developers spend much of their day digging into their systems’ guts, allocating storage to an application, connecting the security system to the user interface and tracking which version of the application is where in the development cycle, among other tasks.

    That’s why a new category of software has emerged, infrastructure-as-code (IaC), that promises to automate system infrastructure provisioning and integration. It frees up time so developers can focus more on frontend solutions rather than backend configuration management. First, though, let’s find out what IaC is and how it helps your organization.

    What is Infrastructure-as-Code (IaC)?

    IaC reduces system infrastructure creation and configuration time through the use of sets of tools, protocols, languages and processes. In conventional data center system management, each configuration entry and change has historically required manual action. Now, technology teams can manage everything in an intelligent, automated manner. 

    With IaC, infrastructure configuration management data is stored within standardized files. The tools monitor and maintain the state of the infrastructure and ensure that the right resources are available whenever an application runs. This saves DevOps teams from excessive manual effort. In essence, large-scale configuration and management become dramatically simplified.

    Development Pain Points 

    Currently, development is a cumbersome process because friction arises in many places. A great deal of progress has been made in speeding up the front end of application development—building the feature set. Still, the process slows down on the back end when the software components are tied to the computer infrastructure. 

    Historically, software was deployed as one large, monolithic block of code. Updates were tedious to make because all of the application’s code had to run, even if it wasn’t involved in the update or bug fix. The advent of containers and microservices enabled developers to design applications that are modular in nature and therefore easier to update, add to or debug.

    However, the application still has to be tied into the existing compute infrastructure. Cloud solutions—like AWS, Google Cloud Platform or Microsoft Azure, for instance—provide companies with quick access to large volumes of computing processing power. But that application software must be told how to access the network, present security information, allocate storage and run code on central servers.

    Because of this, developers spend a lot of time building and maintaining infrastructure-as-code. To do so, they use languages like Python and solutions such as Chef and GitHub. They want to use automation to simplify and accelerate the process, but here, again, they encounter some roadblocks.

    Workflow Bottlenecks

    As applications become larger and components become smaller, developers find themselves mixing and matching software from a wide variety of places. Tracking what’s where in the process is a frequent challenge. Current workflows are often non-intuitive and rely heavily on pull requests. Some solutions try to address the problem with multiple workspaces, but the end result is fragile and nondeterministic code.

    Without mapping of projects to branches or tags, someone who comments on an approved pull request can push unapproved code to production. In today’s fast-paced DevOps work environment, an approval meant for a short-lived test run becomes the current release, creating a ripple effect of unintended consequences that create confusion and defective code, slowing development—often substantially.

    Access Control

    Most tools do not offer access control models. Developers can track a single repository and a single Terraform project, for example, but as the number of repositories and projects grows, ensuring that proper access control features are in place becomes more daunting. 

    Workspace Challenges

    Legacy waterfall and static development projects allowed IT teams to set their system infrastructure up once and forget about it. But in a dynamic environment, microservices come and go in the blink of an eye. New environments proliferate and product teams must continually configure their own workspaces, which eats up time and resources.

    Drift Detection

    Generic IaC platforms don’t provide mechanisms to detect if the infrastructure is drifting. Because so much configuration is done at runtime, differences arise between the desired and the actual state of the infrastructure.

    Drift has many causes, such as developer or machine script contingencies or the lack of system resources. Too much drift results in unstable applications. Catching such problems early in the development cycle is more efficient than trying to fix a production system. But needed visibility is often lacking, and shortcomings become clear only when the system is fully in use.

    Resource Visualization

    Given today’s dynamic environment, developers need clear visibility. They must be able to closely examine what happens as their job runs. Though they offer many advantages, CI/CD platforms deliver little insight into resource usage from a real-time or historical perspective.

    So, whenever performance problems arise, these platforms offer little insight into what the source of the problem is or how to fix it. Consequently, developers’ attention shifts from application enhancements to troubleshooting.

    Lack of Reusability

    Legacy infrastructure failed to let teams reuse tried-and-true system images across multiple environments. To address this problem, teams often create a new image for each version of each application, slowing development time. 

    High Prices

    Setting up each IT environment manually requires time and manpower, which raises costs. Companies must have dedicated teams of engineers responsible for creating the infrastructure. But they also need supervisors and data center resources, which ultimately boost technology expenses. IaC tools centrally manage the process and create the right environment. That way, organizations pay only for system resources that they truly need.

    How Does IaC Improve the Business? 

    Legacy IT practices are expensive, slow and inconsistent because they rely heavily on manual input. Infrastructure-as-code automates much of the infrastructure management process, which delivers a range of benefits.

    Speed 

    With IaC, provisioning and management are lightyears faster and more efficient than with a manual system. The environment’s infrastructure is defined in code. That code-based infrastructure significantly accelerates each phase of the software development and delivery cycle. Consequently, DevOps teams deliver applications faster and at scale while improving software quality.

    Consistency

    IaC deploys consistent configurations constantly and with no discrepancies. Because of this, it enables fast delivery of consistent infrastructure and environments. This reduces errors and overhead while boosting productivity.

    Eliminating Management Overhead

    Infrastructure automation helps to reduce the number of people with hands on the routing parts of the software development process. That’s because human resources aren’t needed in the management of networking and storage. Companies free up personnel to work on more strategic initiatives.

    Reducing Risks and Errors

    IaC helps to reduce risks and avoid human errors. It also protects against runtime issues and security weaknesses caused by missing dependencies or configuration drift.

    Visibility

    Developers version IaC configuration files the same way they do any other source code file. This gives them complete traceability of any changes. In this way, IaC ensures efficient, exceptionally transparent workflows and speeds up the development process.

    Finding an IaC Solution 

    Maintaining legacy infrastructure was a time-consuming, complex process. IaC is emerging as a solution that alleviates much of the stress and pain that developer teams feel daily. Now, they can speed up the deployment and connection of system resources to applications. Developers can become more efficient and enable the company to respond to business demands in a faster, more reliable and more cost-effective manner. The development process no longer slows down the business. 

  • Tips for Better Governance – Techstrong TV

    Tips for Better Governance – Techstrong TV

    David Williams, VP of Porduct Strategy at Quali, and Charlene discuss David’s top tips to better governance in DevOps. The video is below followed by a transcript of the conversation.

    Charlene O’Hanlon: Hey, everybody. Welcome back to Techstrong TV. I’m Charlene O’Hanlon, and I’m here now with David Williams, who is the VP of product strategy over at Quali. David, thank you so much for joining me. Good to see you.

    David Williams: It’s good to see you, too, Charlene. It’s good to be here.

    O’Hanlon: Excellent, excellent. So, lots of stuff going on in the world of DevOps these days, but one of the things that we have been hearing a lot more noise about is governance in DevOps. And especially with just the insane amount of automation that’s going on to help kind of speed up the processes, obviously, it can be pretty easy for things to kinda go off the rails. So I wanna kinda get a sense from you as to what you see happening in the space with respect to governance in DevOps, and how we can kind of maybe move away from the herding cats mindset when it comes to governance.

    Williams: Yeah. It’s been a journey on DevOps. DevOps, when it was introduced many, many years ago as a concept immediately aligned itself with Agile, so it became only the ability to get things out as continuously as possible, and also as fast as possible. And so what happened is that at that time, things were very much managed. It was small teams doing lots of work. As enterprises started kicking in, with DevOps become a serious strategic way of actually delivering applications to their consumers, the amount of people got involved grew. 

      The amount of teams that were involved grew. Even though the DevOps philosophy was small teams, fast moving, application componentry going out into the ether as fast as possible, it stopped being that quite a while ago, and started becoming much more mainstream. So unfortunately, DevOps grew up a lot, and one of the things that became an issue was governance. And governance from that perspective wasn’t really just about security – y’know, SOC 2 compliance, and making sure that your secrets are held and managed efficiently across the pipeline – 

      but it was also around, how do you understand what people are doing? And what is the interaction that they’re having? And why are they having that interaction? And basically it’s a fragmentation challenge, more than anything else. DevOps pipelines are fragmented, and so how do you get control without inhibiting innovation? And how do you get visibility, having lots of teams that wanna innovate in their own areas, but not necessarily give you that full end to end? So that’s the challenge that has emerged. 

      And I think it’s going to continue to do that for quite a while to come, because the applications are getting bigger and more contention-based in regards to infrastructure sharing. So governance is gonna be the thing.

    O’Hanlon: It’s interesting, because I think of it a lot like a company itself growing out there. When you have two or three people working in a company, it can be very difficult to get a lot of achieved without everybody kinda going all in. But as a company grows, you’ve got more people who are doing the work, but maybe they’re not necessarily talking to each other, and not everybody has all the information they need. So you kind of have to kinda build a technology, or at least a process, around being able to get the information out and helping everybody be on the same page. 

      And it kinda sounds like the same thing is happening in the DevOps space with respect to just having a greater understanding. Not just an understanding of the process, but just, I guess, a better way of enabling the process from an understanding and a communication perspective – as well as just making sure that all the right things are getting done at the right time. So what do you guys propose? What do you see that you believe could make a real difference?

    Williams: You used two words that always make me smile when I talk DevOps. I used to be an analyst, talking to a lot of DevOps practitioners, and the two words that I used to run for the hills were “process;” y’know, that was one thing. And so they wanted collaboration, but they wanted to be able to collaborate without being governed in any way. So I think that that process word is one, and anything to do with “abstraction.” So abstraction’s not a good thing. It takes you away from the core. 

      So what I’m saying to you is that any solution that enables you to get greater control on governance and understanding has got to be able to do it without impeding people’s ability to do their jobs. And it’s gotta be something that is subsumed within their normal breathing in and out. It can’t be something that’s another job to do. That defeats the whole point of, anything that’s to do with lean practice is about removing stuff. We’re the software industry, so the way that we make things less complex is by adding something, y’know; that’s the normal inclination. 

      I think DevOps, when you actually use it, whatever you add into the mix has got to be subsumed within your normal day-to-day activities. So it can’t be yet another thing to do. For example, security has gotta be there, but not intrusive. Why I came to Quali was because I’d spent a lot of time with companies where infrastructure, the ability to provision into the cloud, do all the things, was becoming a bit of a mystery. People would assume infrastructure. 

      So the governance around what you’re deploying and how you’re provisioning your infrastructure not only became fragmented through your pipelines – Because you plan something, and then you code it, you build it, you release it, and test it, et cetera. Each one of those is a very defined area. It might be one person; it might be many teams. But each one of those is a very specific thing you have to do. But wouldn’t it be cool if you could subsume testing into your code? Wouldn’t it be cool if you could consume compliance while you’re coding? 

      So you’re coding, and it’s automatically looking at configurations and modifications, and logging that appropriately without you having to add another layer. And I think the same with infrastructure. I think that the companies I’ve worked for before really didn’t know what was going on, and I worked for DevOps companies that were trying to work out, “Okay, so the infrastructure was there; now it’s gone,” or “It’s there, and it’s modified.” 

      Every commit that a coder does is, y’know, a point in time. “So what has happened between that commit and the next commit?” And so all these stages of infrastructure, how do you build that in? So at Quali, what we looked at is providing a method with which you can let people use infrastructure as code practices, configuration management tools. Do what you need, but it gets automatically subsumed into a model. And the model is what you rate on. That’s what applies the policies around access and et cetera. 

      But it should allow the coders, the release managers, the testers to do what they need, but also it provides sort of like a control plane on top that enables you to talk and be governed, but without that Big Brother type of thing looking at you. So I think that’s the magic about DevOps, is if you add something in, it’s gotta be something that doesn’t get in the way of speed. It’s gotta not provide another barrier. It’s gotta do these things. It’s a hard problem to solve, but if you approach it in a way that assumes that people are going to do things, they’re gonna break stuff – 

      People do that; that’s what they do. They do their own stuff. They don’t like being told what to do. So if you can provide as subliminal a level of control over infrastructure that says, “I know how you’re doing it. I’m now going to provide that same capabilities that we used in build to help the testers understand what to build.” So now you’re providing some consistency across, using the control plane. So you’re using your models, and you’re moving them across. But it subsumes what they already use, so you can add it in. 

      Your infrastructure stack and the way that you deploy it is implemented in the model that has policy, access control, secrets integration. And so that’s really the philosophy, is to make sure that we can provide the end-to-end capabilities, accelerate the delivery life cycle, but not impede the practitioners from what they do, but provide the business with visibility which they do not have today. So I would just follow up by saying cost isn’t a governance thing, but unless you govern it, and you understand why you’re doing something, then how do you associate a cost of the cloud, if you can’t associate it with a pipeline? 

      At the moment, cost is associated with instances, and you’re having to group that together. What if you could provide the model with a value while you’re doing it, the application that you’re delivering on? And if you could do that, then you’re subtly applying what they do at the practitioner level with an application, with an owner, with a cost. So that is a way that governance actually comes in with the access, the controls, without being intrusive. We can basically provide all the capabilities there, and enable the business to get better visibility control over all sorts of things, including, for the first time, cost on the end to end. 

    O’Hanlon: Yeah, that makes a whole lot of sense. And in my mind – please tell me if I’m wrong – but it also means that the developers don’t really have to think about, how shall I say? They don’t actually have to learn more about something that would – If they’re moving from, say, a regular kind of a developer motion into more of a DevOps motion, it doesn’t require them to kinda add on a level of expertise that they might not need otherwise. This kinda does it for them, and kind of fills in the blanks for them, and also kinda helps other organizations or other teams within the organization understand the flow a little bit better, which makes a whole lot of sense. 

      And I think that may be part of kinda the governance conversation as well, is if we all have a greater level of understanding about what is happening at any particular time, we can better manage it.

    Williams: Yeah. I mean, secrets management, people hear a lot about that. They don’t quite know what it is when you talk to them about secrets management. I mean, something like a password or a username would be used maybe throughout a process, and it’s a shared piece of knowledge that you don’t wanna expose. So you keep that consistent throughout your environment, but keep the access controls consistent throughout whoever uses that access throughout the pipeline. One thing that we’ve learned is that DevOps isn’t a thing; it’s an interpretation of a thing. 

      The objective is very clear for people, but how they implement it can differ, depending on organization, application, design, et cetera. So what we also have to do is make sure that people with lesser skills – And I don’t mean lesser skills in regards to infrastructure; infrastructure can be cloud provisioned. But what if they have to manage all the networking components and the database instances, the things that aren’t inherent within a cloud-provisioning environment? 

      So we’ve gotta make sure that as developers of different skill levels interact with the infrastructure, we can fill the gaps for them, so they can do what they need to do, but if there’s networking, et cetera that needs to be managed, then the model should be something to give them the guardrails with which to implement that in support of the application requirements to that part of the pipeline. So it is something that we’re working on quite heavily, to make sure that we can get those guardrails in place.

    O’Hanlon: That’s great. And I think about all of the things that have kind of expanded DevOps, if you will, beyond the original parameters, including the implementation of DevSecOps, and the adoption –

    Williams: Yeah.

    O’Hanlon: – and DevNetOps, and all of these things that have kinda piled on top of DevOps. 

    Williams: [Chuckling] Yeah.

    O’Hanlon: It can be very, very difficult for the DevSecOps community, or the DevOps community, especially developers, to kinda have to manage all of that knowledge. So having those guardrails in place, I imagine, is going to be very well received by a lot of organizations, simply because we’re also talking about the dearth of talent in the developer community these days.

    Williams: Yeah.

    O’Hanlon: So anything that an organization can do to help ensure that there is a level of governance around the way their DevSecOps tools and practices and culture and everything is run, I think is a huge benefit. David, thanks very much for walking me through what you guys see happening with DevOps and governance. And hopefully this is an issue that is going to get better as we move forward. To your point, we do tend to throw technologies at problems, and hopefully this is one that will stick, if you know what I mean. 

    Williams: Yeah.

    O’Hanlon: So thanks again for your time. I do appreciate it.

    Williams: Right. Thank you, Charlene. It’s been a pleasure. 

    O’Hanlon: All right, everybody. Please stick around. We’ve got lots more Techstrong TV coming up, so stay tuned.

  • Automation has Become an Absolute IT Necessity – Techstrong TV

    Automation has Become an Absolute IT Necessity – Techstrong TV

    Vijoy Pandey, vice president of emerging technologies and Incubation for Cisco, explains why automation has become an absolute IT necessity. The video is below followed by a transcript of the conversation.

    Michael Vizard: Hey, guys, thanks for the throw. We’re here with Vijoy Pandey, who’s vice president of emerging technology and incubation for Cisco. We’re gonna be talking about all things related to automation. Vijoy, welcome to the show.

    Vijoy Pandey: Thank you, Mike, pleasure to be here.

    Michael Vizard: Right now, everybody is struggling, looking to find people, hire people, retain people, and this was an issue before COVID came, but it seems like it’s much worse these days, and there seems to be no end in sight. So, what is your best advice to folks about how to think about managing talent these days?

    Vijoy Pandey: Yeah, I think that’s a pretty important point you raise there, Mike. Just like everybody else, we’ve been seeing not just news articles around the great resignation, but also we’re feeling that inside of Cisco as well. We know that our partners, our customers, all of us are feeling that pinch, where it’s incredibly important to retain top talent, but it’s also very hard to find talent, given the trends that we’re seeing around us.

    Anything about cloud computing, anything about AI, all of those things. So, we think of the problem in a very different way. We think – and we’ve done a bunch of surveys inside of Cisco – what we realized was given the global nature of events – we know some events unfolding right now – and all of the geopolitical events happening, all the pandemic, people really are realizing that life is too short.

    It’s not a pithy statement here, it’s actually people believe that life is just too short, and they actually want to do meaningful work. They want to make an impact, not just to the team that they are with, nor the company that they’re with, but even a global impact with the work that they do.

    But they also want to grow and learn, and make sure that they are on a path themselves towards meaningfully improving their own lives. So, I think those are pretty important characteristics that we have gathered, and as organizations and as teams we need to harness that desire to make both local impact as well as global impact as well as figure out how to drive meaning into everybody’s personal lives – yours, mine, our teams’, and so on and so forth.

    So, there’s a bunch of things that we are doing inside of Cisco to make sure that this meaning, or meaningful impact, is nurtured. One of the things that we keep on saying is that yes, there is a business to run, but there’s also an avenue we need to give our employees and our teams to innovate, to look at impact beyond the run-of-the-mill business that any corporation needs to do.

    So, typically what we end up doing is we end up taking problem statements that we face, let’s say, as a team or an organization, and we start splitting it into buckets. So, think about buckets that deal with run-of-the-mill business, because that, you need to absolutely take care of.

    But also, there are buckets that you need to innovate around or do meaningful work around. So, we have a portfolio management system inside, at least on my team, that says let’s take 60 percent of your time and make sure that that’s allocated towards run-of-the-mill business as usual projects, and then maybe 30 percent of the time which is future-looking and adding new features or looking at new markets and so on and so forth, and maybe 10 percent of your time is actually towards growing yourself, nurturing yourself, or making some meaningful impact on the global stage. So, that’s actually how we’ve been viewing the problem.

    Michael Vizard: Early on, I think a lot of people were intimidated by automation, and there was concerns that people might lose their jobs. Now, I wonder if we’ve flipped that entirely, where people are saying hey, I don’t really wanna work for organizations that don’t have automated IT process in place because of your points – there’s just things that I don’t wanna spend my day doing anymore. So, has the whole conversation flipped, as far as automation is concerned?

    Vijoy Pandey: I would say the conversation is flipping as we speak, again, yeah, going back to the previous comment, yes, people don’t want to spend time on repeatable tasks, on things which a computer can do better than you and I can do. I have this constant struggle with my kid nowadays as to why do I need to learn grammar when there’s Grammarly? But that’s a different matter.

    But I think it’s a similar mindset going through every individual in the workforce, as to am I really adding value in the workplace, and am I being more productive, which in turn translates is the organization being more productive, which in turn translates to is the economy and the country being more productive.

    So, I think the bump towards productivity is definitely a goal that people are more concerned about, and you hear the other side of this, which is, as you pointed out, is the digitization of the workplace is the digitization of all economy. That’s in progress.

    We’ve called that using various phrases – software eating of the world, or the API economy – whatever term that you want to use. But we are moving towards a world where software is critical in driving everything that we do. Software in terms of automation, or even beyond that, is definitely key to whatever we do.

    Michael Vizard: Not too long ago we all thought we’d be running around by flying cars by now, and that IT would be easy. We’ve seen neither, and it looks like IT is getting more complex than ever. So, my question to you is is IT getting to the point where it is just too complex to manually deal with and we need automation, because we are seeing all these microservices, serverless frameworks.

    There’s all our existing monolithic applications, it’s distributed all the way out to the edge now. Have we reached some sort of inflection point?

    Vijoy Pandey: Yeah, I think to your point IT is definitely way more complex than it was even a decade ago, and I think the crux of the matter is that the way you are building applications and all of the digitization that we talked about is actually taking place in a very, very distributed manner.

    So, as an application developer, first and foremost, we at Cisco like to say that the application is your brand. So, if I am a bank, how my customers interact with the apps that I build defines me, my company, my bank, as the brand. Because if bank A has more capabilities in their software, their mobile app, versus bank B, I will migrate towards bank A.

    So, I think the application is becoming the front end to everything customer experience is all about. So, if you take that notion and play it out, that means the application developer that is developing those applications is under extreme pressure to deliver velocity behind features, do deliver trust with the data of the customers, to deliver uptime and security and so on and so forth.

    So, which means that they are picking and choosing all of these APIs that exist anywhere to make their job faster, easier more secure, more trustworthy. So, if you think about that kind of an environment, as a developer, I’m picking APIs from cloud providers, SaaS providers, I have legacy systems, I have transaction systems, I have edge devices, in the case of a bank. It might be sitting at a branch office.

    I have all of these APIs and systems that I need to interface with just to build one particular app or feature. So, in this kind of a distributed environment, think about the IT – more IT people, right? It’s like I’m trying to build a platform that caters to these kind of demands, and so it is definitely getting more complex, it’s getting more distributed, and every provider in the distributed environment has their own semantics, their own lifecycle management behind their APIs and their infrastructures.

    So, automation – I mean, it’s just humanly impossible to not just deploy but even manage and observe such a highly distributed – and secure, by the way – such a highly distributed system. So, automation is key to doing all of this, a direction automation, to me, is discoverability.

    It’s intent-based, it’s security, which is there’s no longer a perimeter of security, and it’s observability across all of these end points, API to mainframe. That is key to moving forward.

    Michael Vizard: Do you think as we go along that organizations – at least some of them I talked to, it’s almost like a chicken and an egg kind of issue. They have so much legacy stuff that can’t be automated that they can’t invest in automation, and they can’t necessarily buy all new platforms tomorrow. So, how do we get over the automation hump to kind of get from point A to point B without necessarily breaking the IT budget?

    Vijoy Pandey: Yeah, that’s a great point. I think some of that, I’ve dealt with in my previous lives as well. The way to do that is – and there’s no questions about it, that for some period of time, there is going to be more chaos and more money spent than before that time or after that time.

    So, we just have to acknowledge that as vendors, as customers, as organizations, that for some period of time, there’s going to be turmoil. That’s the way, at least, I’ve handled it in the past, where you say let’s put milestones, where we say maybe we look after a pocket of the organization, and say we’re going to build this out in a highly automated, distributed, cloud-first, API-first manner.

    This part of the application, this part of the infrastructure goes out and does that. To do that, we’re going to do a clean slate process here. So, we’ll think about – we won’t get ourselves bound to organizational boundaries, skill set boundaries – that is very hard to do. Most people’s jobs, peoples’ skill sets, organizational structures, all of that.

    But we have to try in some pockets, and then give ourselves strict guidelines and metrics, and try and meet those metrics. If we meet those metrics, the resulting outcome is that I will be spending less moving forward than I’m spending today.

    So, I might spend a little bit more in getting maybe SREs or software engineers, and marrying them up with IT folks or architects, and getting something going. I’ll give myself a timeline of let’s say a year or 18 months; it cannot be longer than that, otherwise it’s a science project. 

    But after those 18 months, these are the metrics I’m going to come back to. I’ll get X percent improvement in productivity, and Y percent reduction in head count, let’s say. We shall measure ourselves to that objective, and I’ve seen that work.

    If it works – and start with something simpler. But go all the way, don’t go 80 percent. Let’s not do 80 percent automation or 80 percent of the use cases. Hundred percent automation, 100 percent of the use cases for this pocket of your environment, and then grow that out over time.

    Michael Vizard: We, of course, hear a lot about artificial intelligent and AI Ops. Are the machines gonna save us from ourselves? What’s the reality of AI?

    Vijoy Pandey: To me, AI is a tool, just like all of the other tools that we’ve talked about, which is we’ve talked about automation or infrastructure as code, we’ve talked about cloud native and serverless and microservices. Each one of these tools, programming and scriptings tools.

    So, all of these tools bring in something to the table, and infrastructure as code brings in consistency and intent-based infrastructures, and serverless and microservices, or cloud-native, bring in immutable architectures.

    The fact that you can grow and scale out and do all of those things, and availability goes through the roof and velocity goes through the roof. Similarly, AI and ML bring in insights to the table where it was humanly, again, impossible to do that just by looking at the vast variety of data and the deluge of data coming our way.

    If we could just take all of that – and it’s all unstructured. So, if I could take all of that and somehow put it in a structured database and somehow make sense of it as an individual, sure, we don’t need the AI or ML. But if I need to take all of that unstructured data and make some sense out of it, there’s no other way out, and I do need AI and ML.

    I mean, I do need some statistical mechanism first, to figure out and make some sense out of it. I do need AI/ML, maybe deep-learning, to make some sense out of it. But then I need to marry that tool with subject matter expertise, I need to marry that tool with what I as an individual, or as humans, we’ve learned in the past, because AI doesn’t have that context.

    Yes, we can do learning, all sorts of learning, but it doesn’t always work in all kinds of environments. We need to marry that with subject matter expertise, with some boundary conditions, and then we need to marry it with actions. Because AI/ML is gonna give us insights, but somebody needs to take actions on the insights.

    So to me, it’s a tool in the toolbox, and we need to leverage it when it makes sense. We need to rely on subject matter expertise when it makes sense, and eventually things will get to a point where we need to go to the next level of abstraction. 

    So, it’s always a game of moving that level of abstraction higher and higher, and that’s what we are after. So, maybe someday there will be flying cars and robots that will take all our jobs, but not right now.

    Michael Vizard: How do I bridge this divide in IT that maybe we don’t talk enough about, but there’s clearly a DevOps community that’s very programming-centric. The use APIs. Then there’s traditional IT administrators that use ITSM tools. They tend to be more graphical in nature. As we move forward with automation, do you think that those two camps are gonna converge, or how will this all play out?

    Vijoy Pandey: In my opinion, I think they have to converge. I think each one of those pieces have their space in the hierarchy of things. One of the things I learned pretty early on in one of my previous lives is what good is a dashboard if you cannot take an action on it?

    So, if you think about all dashboards that exist today, eventually, there is data being presented in a certain way about a certain problem, so that somebody can look at that and take an action. If you think about infrastructure as code, it’s exactly that.

    It’s basically let’s remove the dashboard, and let’s figure out what goes – triggers that, that allow me to take those preset actions. Let’s try and remove the action of a human looking at a dashboard, thinking about it, and taking some action on it.

    Now, again, I’m describing utopia, but I think that’s where, as engineers, as operations folks, we all need to move towards. To me, dashboards are a good start when the problem is less well understood. As we understand the problems better, we need to move towards code.

    Dashboards will still have their place in helping us understand more and more complex problems, but as problems are better understood, they need to move towards code. And that’s the way at least I think through it.

    Michael Vizard: All right, so automation needs to be our new North Star, as it were. Hey, Vijoy, thanks for being on the show.

    Vijoy Pandey: Thank you so much, it was a pleasure, and I enjoyed our conversation.

    Michael Vizard: All right, back to you guys in the studio.

  • Leverage Empirical Data to Avoid DevOps Burnout

    Leverage Empirical Data to Avoid DevOps Burnout

    Burnout is possible in any industry. But it seems to be especially rampant in tech, a space bursting at the seams with inflated expectations and ever-accelerating trends. Nearly 60% of tech workers said they were currently feeling burned out at their workplace, found a study by TeamBlind.

    Limiting burnout and supporting employee mental health is essential, not only for the workers’ sake but for the business as well. During this period, which commentators call the Great Resignation, if employees don’t feel respected, they will likely seek new opportunities elsewhere. Ensuring the needs of engineers are met is a vital target amid an ongoing cloud skills drought, a talent gap to which the DevOps space is certainly not immune.

    I recently met with Eric Mader, senior solutions architect, AHEAD, to understand the root causes of burnout as it relates to DevOps. According to Mader, burnout can be caused by many factors, including work-life imbalances and conflicting egos in the workplace. But above else, burnout is most often caused by management setting unrealistic targets that aren’t based on empirical team performance data.

    What Factors Contribute to DevOps Burnout?

    So, what can cause burnout in DevOps? First, the workplace culture can significantly contribute to burnout. In most remote working models, explained Mader, it isn’t easy to ascertain the correct work-life balance. For example, there may be an unwritten expectation to immediately respond to Slack messages or quickly get back to emails after-hours or over the weekends.

    An “always on” attitude can lead to unhealthy work habits. And while this isn’t necessarily unique to DevOps, it is sometimes unavoidable due to the high connectivity required to perform the job. Another potential contributor to burnout is the team dynamic, noted Mader. “Every team seems to be unique in its own way,” he said.

    Although Agile and DevOps involve very technical work, at times, ego can get in the way of a healthy team dynamic. For example, in many tech teams, there is often a “guru,” such as a senior developer or tech lead, that everyone goes to for advice, Mader said. This person might end up doing most of the work (both technical and interpersonal) and usually feels the most burned out. Or, other wanna-be leaders might take on an unnecessary command-and-control role.

    Personality differences with coworkers could contribute to burnout, but other factors like dealing with toil or managing legacy technical debt can be just as tiresome.

    ‘Move Fast And Break Things’

    According to Mader, perhaps the biggest contributor to burnout is the insistence from executives that tech workers consistently outperform their previous output—especially when past team performance data does not support these goals. Setting achievable goals is great, but when deliverables are out of scope, it’s a recipe for burnout.

    Part of the pressure to increase velocity is the constant illusion that “faster and more is better,” said Mader. Value is often attributed to the number of items achieved in a sprint instead of the quality of each point. For example, a team might deliver 200 story points of work in a sprint, but only 30 of these points carry a significant amount of value. Or, a leader might see a team accomplish 60 points of work in a sprint and then encourage 200 in the next one.

    These sorts of high expectations suffer from basic capacity problems. And the added stress quickly realizes itself as a burden, contributing to burnout, explained Mader. He encourages technical leads not to force more work but instead, “focus on the highest-value work based on the empirical data you have.”

    Five Tips To Avoid Burnout

    Mader shared some tips on how to avoid burnout in DevOps teams. Here are some simple yet effective ideas to maintain happier engineers and a more productive, healthier team relationship.

    Snuff out the always-on attitude. Organizations need to walk the talk when it comes to mental health awareness. Therefore, ensure employees know that their life boundaries are respected. Encourage this by honoring time off for holidays, paternity/maternity leave or personal vacation. Discourage emailing on the weekends and encourage turning on ‘Do Not Disturb’ alerts on team chats and work email accounts.

    Prioritize what gets automated. There’s a misnomer that any automation is good automation. In fact, over-prescription of software development automation can lead to some negative consequences. “I’ve seen many DevOps teams that focus so much time on automating low-impact business processes,” said Mader. Instead, prioritize automation projects that affect more people and those that would be continually used.

    Try out the ‘Fist of Five.’ Ensuring everyone’s voice is heard helps avoid a toxic working environment. One tactic to quickly gauge reaction in team meetings is to use what Mader calls The Fist of Five: One if an idea is totally off base and five if an idea makes total sense. This simple practice could ensure everyone gets to contribute input on new projects.

    Take the quietest person in the room aside. Only the squeaky wheel gets the oil, Mader noted. With that in mind, take aside the quietest person post-meetings to gather their input—you might be surprised. Junior folks who may be less apt to speak their minds typically have a less-institutionalized mindset and could suggest more innovative ideas.

    Use empirical data to inform future goalposts. Finally, Mader encouraged DevOps teams to “use empirical evidence to back up on where you’re going in the future.” Specifically, in the DevOps sphere, empirical data takes the form of metrics like mean-time-to-recovery, or SLAs, SLIs and SLOs.

    Remember that even if a team did accomplish a certain number of goals in a sprint, it doesn’t mean they can produce the exact same results the next time around—they could have been pushed to their limit to produce past results. Thus, managers should aim to move the needle on metrics progressively, not in giant strides that upset morale.

    Take an Opinionated Approach to DevOps

    Nowadays, DevOps teams are spoiled for choice—there are countless tools on the market and an endless stream of best practices and methods. Even within a single company, the DevOps toolset may vary considerably from team to team. To help make sense of it all, Mader encouraged DevOps leads to simplify the transformation journey by setting guardrails and adopting a more opinionated approach around what tools and processes you take on.

  • Applying TestOps to Cultivate a Quality Culture

    Applying TestOps to Cultivate a Quality Culture

    Quality assurance is a universal concern, especially when it comes to application development. Enterprise applications are faster and more reliable than ever before. However, in addition to performance, application developers also must consider data security, regulatory compliance and cost-effectiveness, among other things. Organizations are embracing quality engineering (QE) as a foundation to create a software quality culture to address these challenges. Creating a quality engineering or TestOps environment is not simple, especially when QE departments must test for hundreds of applications across multiple enterprise environments. Whether it is a solution provider or a multinational corporation, applications often need to be tested across platforms, industries and geographies.

    TestOps encompasses a broader concept than just QE. With TestOps, there are testers and engineers using automated and manual testing to validate software for flawless application performance. In enterprise DevOps, the TestOps team serves as the bridge between development and deployment since they have a good understanding of the applications and a better perspective on the context and variables that can affect performance.

    Continuous testing is an integral part of any software development cycle. Quality engineers must be brought in early as part of Agile development to keep development on track and head off any problems that could escalate or become harder to resolve later. In Agile, testing becomes more crucial and frequent to ensure flawless performance.

    It pays to think horizontally when setting up a TestOps approach. For example, testing should be performed across different vertical environments such as media, manufacturing and finance. The goal is to create a test environment that can accommodate various industries.

    TestOps Across Environments

    In the age of SaaS and universal data access, it is essential to test across multiple platforms. In addition to ensuring enterprise performance, mobile web and remote data access must be tested. For streaming services, it is vital to test across mobile platforms, over-the-top (OTT) platforms such as Roku, Apple TV, Android TV and Fire TV, and even gaming consoles like PlayStation and Xbox.

    When shipping software updates, geography matters (especially when shipping across time zones), as one may never know when the software will be used or have control over the environment where they plan to use it.

    For instance, consider the challenges a leading mortgage lender faced with mobile application testing and how geography was a factor. First, Google Analytics was used to assess specific device and user requirements and ascertain performance in various parts of the United States since different regions use different carriers. Testing was then done across mobile platforms such as iOS, Android and combinations and across carriers since most banking and mortgage applications required one-time passwords (OTPs) and SMS-related capabilities. Further tests were implemented for new identity authentication strategies such as biometrics and facial recognition. Finally, performance testing by geography was conducted to determine whether a new feature or version would create issues.

    Testing for Geographies and Regional Compliance

    Localization testing is important since every region and every country has unique compliance issues and laws. To ensure proper localization, it is essential to perform complete testing, including lower environments and production systems.

    In any TestOps environment, the philosophy of continuous testing should already be part of the design environment, including understanding the nuances of production. Release readiness does not mean there are only a few known bugs in certain regions. It means the application has been tested for compliance in every geography where it could possibly be utilized.

    Testing Across Industries

    Testing application performance in specific industry settings brings along its own set of unique challenges. In manufacturing, for example, each sensor in an IoT device must be modeled against a conformance specification to determine if it can be automated using simulations. In media, one needs to test for different platforms, content management systems and business workflows.

    To accelerate time to market, one would want to support a dual-shift development lifecycle to gauge the consumer experience and get continuous feedback and also need a test environment with multiple automation tools such as Selenium, Appium, Cypress, Tricentis, UiPath, Test Complete and Winium.

    Artificial intelligence (AI) plays a larger role in testing, shortening time to market, and ensuring superior quality software. AI-driven test tools are optimized for domain-specific applications, such as testing for industries like media, manufacturing and fintech. AI can fill in the gaps to achieve optimization and offer a quicker turnaround with limited manual intervention.

    Any organization cultivating a TestOps culture needs to have specific technologies and tools in place. When assessing your testing technology stack, it is important to create an end-to-end testing ecosystem. This means the environments must be inclusive and capable of incorporating testing for unique applications, environments and scenarios. To support the needs of today’s Agile application development environment, one must plan for an agile testing infrastructure to keep development on track.