devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • Is AI Taking Over DevOps Functions?

    Is AI Taking Over DevOps Functions?

    Artificial intelligence has taken the tech world to a new level of automation. Today, almost every specialization demands the intervention of machine learning to develop AI technologies that help businesses do more with less time and resources. Still, some organizations question whether leading with AI is a good investment. 

    For DevOps, the answer is a resounding yes. AI can enhance DevOps practices to accelerate the pace of software releases, helping businesses achieve continuous delivery. This enables programmers to release software about 10 times faster and allows programs to be reviewed before they are released. 

    AI also has improved DevOps culture, from enabling more efficient decision-making processes to code quality enhancements and automated operations. So, let’s take a deeper dive into how AI impacts DevOps.

    Recent Changes in DevOps

    Monitoring and managing all the data generated in DevOps environments involves a high degree of complexity, making it difficult for teams to collect and use data effectively. In addition, the volume of data any given team may receive can reach well into exabytes, so AI tools provide much-needed assistance. After all, analyzing massive data sets manually would take too much time for a human and won’t meet the demands of modern businesses.  

    What’s more, surveys show that 57% of developers in the United States have less than five years of experience. For this reason, software testing must be an ongoing and in-depth process to check and double-check that no vulnerabilities impact the security of the code. AI can supplement manual review to increase speed and accuracy in finding bugs. 

    That being said, integrating AI into DevOps functions doesn’t mean developers aren’t necessary, even though their roles have evolved considerably. Backend developers are involved in the development and operations sides as they take on ops tasks and tests, especially around cloud-based infrastructure and cybersecurity. And frontend developers will always be needed for the technical support of creative design—which no one can do better than humans. 

    In today’s dynamic and fast-paced world, developers need to release code faster than they used to while the operations teams ensure that the existing systems function with minimal disruption. This partnership is easier when integrating AI, as it makes the collaboration between development and operations teams more efficient. AI-powered systems provide a unified view into systems and glitches in the complex chain of DevOps. 

    Is AI Taking Over DevOps?

    Short answer: No. AI is supplementing human activity in DevOps to make things more efficient and secure. 

    Due to the complexity of DevOps requirements, people have been looking for a more automated solution that could assist humans in performance and accelerate the processes. What’s more, you can expect to pay around $60 an hour for an experienced backend developer in the U.S. This means using AI to automate the grunt work can be a big cost saver as well.

    Here are a few ways AI transforms DevOps functions:

    Enhanced Security

    Distributed denial of service (DDoS) and related attacks have become a widespread threat to the security of websites and online services these days. AI-powered tools have the ability to identify and mitigate these threats. In addition, AI can help in the pivot to DevSecOps in teams still using traditional DevOps.

    AI security tools detect anomalies and threats based on real-time data and by analyzing past behavior. In this way, AI has played a vital role in protecting many organizations, including schools and colleges, from cyberattacks by supplementing human security experts.

    Improved Data Access

    The massive amount of data that needs to be collected, organized and analyzed in DevOps is beyond human capabilities. However, AI has resolved this issue by compiling data from multiple sources and organizing it for analysis. This drastically improves and accelerates the process of DevOps.

    Rapid Software Testing

    Software development and software testing has accelerated with the integration of AI. As a result, various testing types used by DevOps, such as user acceptance testing, regression testing, and functional testing, are more efficient and accurate than ever.

    These tests produce a large amount of data that would take humans ages to identify and collect manually. With the help of AI, identifying patterns and coding practices that lead to errors and vulnerabilities has become convenient and fast. This data can then be used later by the DevOps team to increase their efficiency and improve development practices. 

    Timely Alerts

    DevOps teams sometimes struggle to react and respond to the alerts during incident response. With no priority tags attached, it becomes challenging for the team to prioritize incidents effectively. A more efficient, comprehensive alert system should be capable of spotting flaws instantly and marking the more severe ones as higher priority. That way, the team can tackle the issues more systematically and resolve them without failure.

    AI can assist in prioritizing the responses to alerts based on certain factors, including the intensity of the alert, the source of the alerts and past behavior. This leads to efficient management in situations where systems get flooded with massive amounts of data and have to respond immediately. AI-powered alerts are particularly important in DevOps security, so this is a critical benefit of AI in the discipline. 

    Superior Implementation Efficiency

    DevOps teams are often burdened by the managerial tasks of a rule-based environment which decreases their time spent in more innovative and creative areas. When AI takes over, these tasks become self-governed, increasing efficiency and reducing human intervention. AI systems can work on their own, sparing humans from these tasks to focus more on innovation and creativity.

    This way, AI has turned the DevOps into self-governed systems that no longer require the rule-based, human management of analysis. This should resolve the complexity of research that DevOps teams were struggling to achieve and enable faster adaptation.

    Feedback Loops

    The primary function of DevOps is to use monitoring tools to gather feedback from every stage of operations and then use that to improve software development and delivery processes. Performance monitoring tools use machine learning to gather information such as performance matrices, log files, datasheets, etc. The feedback collected is then used to track potential issues in advance and make suggestions to resolve them accordingly. The application is then altered by applying these suggestions to make it more efficient.

    Artificial intelligence is driving increased efficiency in DevOps functions. However, it would be a stretch to say that it is taking over, as there are still many areas that require human intervention. The idea behind AI integration is not to change the players but to change the game.

    Even though AI is assisting humans in accelerating software development, developers still lead many aspects of the craft. But it also seems impossible to perform DevOps functions at the required pace without the integration of AI. So, if you wish to release software at a pace that keeps up with modern tech trends, you need to incorporate AI in your DevOps functions.

  • Sentry Integrates Code Observability Tool with GitHub, GitLab

    Sentry Integrates Code Observability Tool with GitHub, GitLab

    Sentry this week extended its namesake observability platform for code to provide tighter integration with the DevOps platforms provided by GitHub and GitLab.

    Application development project managers can now assign issues to DevOps teams by integrating directly with the CODEOWNERS file in GitHub or GitLab. Error messages, for example, can now be automatically routed to individuals or teams that own the code via notifications in Slack.

    IT teams can now also identify and prioritize issues based on user impact by setting alerts based on when an issue exceeds a certain percentage of user sessions within a specific amount of time.

    Sentry CEO Milin Desai said that the percentage-based approach to alerts reduces the overall noise being generated in a way that enables DevOps teams to better prioritize their remediation efforts.

    Finally, Sentry has added an integration with the Okta identity management platform to make it possible to automatically provision and de-provision users and teams using the system for cross-domain identity management.

    As more responsibility for managing the application environment shifts left toward developers, there needs to be a way to streamline communications between developers and the rest of the IT operations team, noted Desai. Most developers today already spend less than one-third of their time writing code.

    The Sentry platform is already used by more than 75,000 organizations to enable developers to better observe and analyze code behavior. The company is now moving to bridge the divide between developers and site reliability engineers (SREs) by streamlining communications and workflows, noted Desai.

    As the responsibility for managing applications shifts left, developers don’t necessarily want to use the same tools that IT operations teams have relied on for years. Most developers want to be able to troubleshoot their applications with minimal interaction with IT operations teams.

    Historically, IT operations teams have relied on application performance management (APM) platforms to manage IT operations. Those platforms are now evolving into observability platforms that promise to provide more context by correlating events as they occur across both applications and IT infrastructure platforms. That approach, however, typically requires developers to insert agent software into their applications. Once inserted, that agent software needs to be maintained and updated, added Desai.

    The Sentry approach provides developers with a tool for collecting data from an application in a way that consumes just tens of kilobytes of memory after being embedded directly within an application. That approach eliminates the need to rely on much larger agent software to achieve observability of application code. Developers can then diagnose and possibly remediate issues in code without ever having to create a ticket asking a DevOps team to investigate an issue.

    Regardless of how developers go about instrumenting their applications, most everyone in IT agrees that the current state of observability needs to improve. The issue now is determining how best to achieve that goal in a way the benefits both the DevOps team and the developers that are now being asked to not just write code but also manage the entire application life cycle.

  • Empowering Teams to Make Mission: Impossible Possible

    Empowering Teams to Make Mission: Impossible Possible

    “Your mission, should you choose to accept it….”

    That’s the way the original Mission: Impossible TV show and, later, the movie series drew us in. The bad guy’s operation was explained. The goal was clearly defined; infiltrate and destroy. And although the task at hand was extremely difficult and fraught with unknowns, the team of good guys was briefed and everyone understood what they were doing and why. Cut to the good guys succeeding despite the odds being stacked against them.

    For most of us in product development, I think we’ve all faced—or are facing—our own impossible mission. We’re all working on “urgent” projects against some pretty tough odds. Except, unlike the old TV show, we often lack a clear goal. Sure, we’ve got our best team members on it—but does everyone understand why we’re doing what we’re doing? What, exactly, is our mission?

    No Dossier, No Chance

    In the frantic rush to “digitally transform” by churning out new products and features, the mission (if there ever was one) is often lost. Without context, developers can feel like order takers instead of outcome producers. This dynamic is a ticking time bomb; not just in terms of value delivery and business success, but for developer happiness, too. With a worldwide developer shortage and, more importantly, the fact that 80% of developers are reporting burnout, something needs to change. And fast.

    Improving the software developer experience must be a priority. And a clear mission is a great place to start. When we’re not mission-driven, our chances of being innovative, experimenting with new ways of work and empowering ourselves to be at our best are severely limited. 

    Liberated Mercenaries 

    By adopting a mission-oriented approach, development teams gain purpose. Bringing them in on the business problem that needs solving can motivate them and stimulate their creative thinking. And the cherry on top? We can even help reduce their workloads by removing meaningless blackhole tasks. We begin cultivating a work environment that helps realize the “joy;” part of one of Gene Kim’s Five Ideals. 

    For those overseeing DevOps or other continuous improvement initiatives, we can improve our odds by creating our own Mission: Impossible-like teams. To cite John Doer, venture capitalist, we can build teams of missionaries instead of mercenaries. Folks who are motivated, passionate and outcome-driven. Those galvanized by the excitement of knowing that their efforts will impact the business in a big way will inspire greatness. 

    It is the leader’s role to articulate the mission. When practitioners are empowered to use their creativity for problem-solving, innovation is unleashed and there is little room for misguided, wasteful activities.

    Preparing your Mission Dossier

    You are not alone if your answers to the following are no better now than they were five years ago:

    • Are we prioritizing our customers’ problems first?
    • Are we enabling customers to pull value rather than pushing what we believe is value?
    • Are we providing abundant opportunities for customer feedback and pivoting with learnings from failures?
    • Are we routinely surprising and delighting our customers with exceptional value?

    These are the kinds of questions that indicate how well we understand our mission. If we are aligned with customers, then, by definition, we are aligned with the business since businesses only exist to serve customers.

    So then, how do we articulate the mission so our teams clearly understand their purpose? Here are three steps to get you started.

    Step One—OKRs

    We must emphasize our focus on solving problems for our customers, both internal and external. The objectives must be clearly stated, as they were for Mr. Phelps. And, likewise, we must communicate the sense of urgency that this mission encompasses for the fleeting window of opportunity that exists before customers seek alternative solutions. 

    One highly effective approach is to employ objectives and key results (OKRs) and an OKR tree so that everyone, from the trenches to executive leadership can have line-of-sight to the purpose of their work and their role in the mission. Since DevOps Enterprise Summit 2020 in Europe, OKRs have emerged as a hot topic as leaders seek to translate urgent corporate objectives into meaningful goals for their business and technology teams.

    Step Two—Focus on Flow of Customer Value

    We must set the stage for the successful delivery of customer value. Our processes must be aligned with real customer needs while simultaneously capable of delivery excellence. The best way to do so has been proven out for over 40 years now, i.e.: adopting a continuous improvement approach to the activities which make up the value stream from customer request to fulfillment. 

    That means capturing customer needs via our OKRs and accelerating the flow of work across the value stream against those OKRs. To do that, we must have a means to systematically and continuously remove impediments to flow across all key stages, before and after development

    This approach encourages everyone—top-down, bottom-up—to become single-minded and obsessive about our mission.

    Step Three—Tracking The Mission

    With the mission clear and the teams aligned, we must also be prepared to react and preempt uncertainty. No mission runs perfectly. But we can put literal measures in place to identify danger ahead. Combining team performance metrics, such as DORA, with higher-level value stream management (VSM) metrics that focus on the flow of business value across the whole system can help everyone in the field and in the control room communicate effectively. For instance, if work upstream is holding our developers back from cracking the code, VSM metrics can help direct leadership to send support into the business analysis room.

    With mission alignment and means of communication, accepting a mission becomes enticing for everyone involved and the chances of success improve dramatically. 

  • DevOps World: Forget Doc, it’s All About Dev

    DevOps World: Forget Doc, it’s All About Dev

    This year for DevOps World, we’re heading back to the future. Doc Brown won’t be there with his souped-up time machine, but plenty of devs will. As the world is coming to realize, developers are keeping everything running, from the apps we use on a daily basis to check the weather, bank online or order food to the self-parking features on autonomous cars and artificial intelligence that’s augmenting human intelligence. Software and technology are the future, which means the future starts with the creative mind of the developer. 

    The theme of this year’s DevOps World show is “Building the Future of Software, Together.” For developers, this cannot ring more true: Across the globe, dev teams are working to code new software pipelines that will power the world’s next great innovations. We’ve already seen voyages into space and breakthrough life-saving medicines this year—what more do developers have in store for us?

    As we look ahead to DevOps World 2021 on Sept. 28 to 30, we are putting the focus on developers by offering a number of sessions to help sharpen their skills and prepare them to build the future of software. In particular, we are looking at some of the recent challenges that have been affecting code, including cybersecurity, scalability and CI/CD best practices. 

    On the security front, we’ve seen a number of cyberattacks dominate news headlines this year. As a result, companies are hyper-focused on securing their software pipelines and ensuring business and customer data are safe. Developers now need to double as the first line of defense for their organizations, ensuring their code and pipelines are as secure as possible. We’ve geared a number of sessions towards helping developers sharpen their security chops—from CloudBees customer success stories to tips for building more secure code. 

    As DevOps professionals know, software today is about continuous everything. We’ve designed a number of sessions to share best practices for CI/CD, progressive delivery and balancing speed, innovation and safety. Imagine if the DeLorean didn’t need the plutonium and just ran continuously—that’s what we’re hoping developers will learn and take away from these sessions!

    Lastly, scalability has always been a hot topic in tech. Startups continue to pop up everywhere and as these companies grow, they need to scale accordingly. We want developers to be scaling pros when they need to be, so we have a number of sessions (including keynote presentations) on scalability. 

    DevOps World is the event for developers looking to sharpen their skills, learn new abilities and network with other dev professionals. As we look to build the future of software together, the people making the magic behind the code will be driving changes and innovation, and we want to celebrate that.  While Doc Brown may be a fun character who goes back to the future, it’s dev where the future truly is. Register for DevOps World now! 

  • Transposit Survey Shows DevOps’ Impact on CIO, CTO Relationship

    Transposit Survey Shows DevOps’ Impact on CIO, CTO Relationship

    A survey of 415 CIOs and CTOs conducted by Transposit, a DevOps process orchestration platform provider, finds that as organizations continue to adopt DevOps processes more widely, friction between these types of senior IT leaders is rising within many organizations.

    As a C-level role, ‘CTO’ first gained traction in the wake of the dot com boom. In more recent years, many traditional enterprises added CTOs alongside CIOs as they realized how dependent the organization had become on software. In the wake of the COVID-19 pandemic, there is now more focus than ever on digital business transformation initiatives that are dependent on the timely delivery of software.

    Transposit CEO Divanny Lamas said that as more organizations’ structures revolve around product teams, there are a lot more CTOs. In some cases, CIOs are reporting to CTOs.

    The majority of organizations surveyed have both roles. A full 83% of CIOs said there is also a CTO at their organization, while 86% of CTOs said there is also a CIO. Just under a quarter of respondents (24%) said they perform both roles.

    At the same time, 98% of survey respondents reported increased focus on digital transformation within their organization in the past three years, with three-quarters (75%) noting that focus has increased since the start of the pandemic. A full 85% report their budget has increased over the last three years.

    Not surprisingly, 92% of respondents said that their responsibilities have changed in the last three years, with over half (56%) reporting their role changed significantly. Both are now graded according to key performance indicators (KPIs) related to managing revenue, with 66.3% of CIOs and 78% of CTOs saying their position is now either directly involved in or required to lead a business strategy.

    But only 34%, or just over a third, said that ownership of most aspects of digital transformation initiatives is a shared responsibility between the CIO and CTO. Specifically, just under a third (32%) noted ownership of software development in their organization is a shared responsibility. A full 88% of respondents apply DevOps practices, and 83.4% believe that DevOps is important to digital transformation at their organization. Almost all (98%) plan to expand their DevOps practice, and 83% already have a plan in place to do so.

    Lamas says software engineering-first principles are changing not just how applications are built but also how IT teams are structured in a way that advances innovation.

    More than a third (35%) of respondents said ownership of software operations developed in-house is a shared responsibility in their organization, while 28% said ownership of managing SaaS or operating commercial off-the-shelf (COTS) software in their organization is a shared responsibility. A third (33%) also noted ownership of the technology for the external customer experience in their organization is a shared responsibility.

    A full 95% of survey respondents also confirmed that the performance of their team is measured by key performance indicators (KPIs) related to revenue, specifically revenue growth (75%), rate of innovation (60%) and uptime (54%).

    It’s too early to say how the relationship between CIOs and CTOs will evolve, but the one thing is clear: The pressure for these roles to collaborate more effectively is on the rise.

  • Sonatype Report Shows Spike in Supply Chain Attacks

    Sonatype Report Shows Spike in Supply Chain Attacks

    Sonatype today released a report that finds there has been a 650% year-over-year increase in supply chain attacks aimed at upstream public repositories. Cybercriminals hope to compromise these repositories by injecting malware into software components that many organizations might be using, according to the report.

    The seventh annual State of the Software Supply Chain Report is based on an analysis of 100,000 applications and 4 million component migrations made in the last month alongside a survey of 702 software engineering professionals. Specifically, the report focuses on demand and security trends associated with the Java (Maven Central), JavaScript (npmjs), Python (PyPI) and .Net (nuget) ecosystems. The top four open source ecosystems now contain a combined 37,451,682 different versions of components, a nearly 20% increase year-over-year.

    Over the course of 2021, developers also downloaded more than 2.2 trillion open source packages from the top four ecosystems, a 73% increase over the prior year.

    However, despite the huge available supply of open source software, use of this software is primarily concentrated in a small number of popular projects. Production applications are only using 6% of available open source projects.

    Not surprisingly, popular projects are more vulnerable to compromise. A total of 29% of popular project versions contain at least one known security vulnerability. Conversely, only 6.5% of projects that are not widely used have vulnerabilities.

    Stephen Magill, vice president of product innovation for Sonatype, said popular open source projects tend to have better security practices in place because of the larger number of maintainers and contributors working on the project. The paradox is that less-popular open source software is not likely to be targeted by cybercriminals who tend to focus on targets that represent the best chance to infect downstream applications, said Magill. Popular open source projects were 2.8 times more likely to contain vulnerabilities, the report found.

    Projects with a faster mean-time-to-update (MTTU) are also generally more secure. They are 1.8 times less likely to have vulnerabilities, the survey found.

    The report also found developers make suboptimal choices 69% of the time when updating third-party dependencies, even when newer versions of projects are generally available. Developers may be avoiding those updates because they may be perceived to be not as stable as they would like. However, those updates generally contain the latest security patches needed to address a recently discovered vulnerability. Commercial engineering teams only manage 25% of the components they use, leaving the majority of their open source dependencies stale and susceptible to increased security risks, the report found.

    Finally, the report suggested there is a greater need for automation. Equipped with intelligent automation, a medium-sized enterprise with 20 application development teams would save a total of 160 developer days a year. Sonatype estimated that could save organizations $192,000 a year.

    In general, Magill noted that the security of software supply chains should continue to improve in the wake of an executive order issued by President Biden. In the meantime, however, most organizations would be well-advised to start reviewing the components and dependencies that make up their software supply chain sooner rather than later. After all, chances are good that cybercriminals are already doing much the same thing.

  • Choosing the Right Metrics for DevOps Adoption

    Choosing the Right Metrics for DevOps Adoption

    DevOps adoption has skyrocketed in recent years. Its impact on traditional development platforms has long been well-documented, but in more recent years it has started to influence low-code platforms, too. In its recent survey on the state of Salesforce DevOps, Gearset found that 69% of teams are already using source control and another 20% plan to start this year. Additionally, 45% of companies have implemented CI/CD, and that number is set to increase by 33% this year. However, for high-performing developer teams to recognize the benefits of DevOps and strive for further DevOps adoption, they must use the right KPIs.

    While it can be tempting to take a ‘big bang’ approach to DevOps adoption, teams see more success when they take it one step at a time, with each stage building on the last.

    That said, to experience the full ROI of DevOps adoption, it’s still critical to see the journey through to the end. It’s not uncommon for teams who begin the process of DevOps adoption to stop halfway through or, worse still, lose motivation and halt the process entirely. DevOps adoption is a process full of technical and cultural change, and one of the most common causes of its failure is an emphasis on the technical at the expense of the cultural.

    The best way to tackle this is to encourage teams to take a product mindset and apply it to DevOps adoption. Simply put, rather than tasking your teams with “adopting CI” for instance, they should be asked to solve a specific business problem. For example, “When we finish work on a story, it takes weeks for it to get released!” It sounds obvious, but focusing teams around business impact rather than on individual tasks makes sure that there’s a direct correlation between the work that’s happening and the desired business outcome. This is the crux of adopting a DevOps mindset, rather than a purely technical and perfunctory approach to DevOps adoption.

    One of the key things a team can do to drive this type of change—and to avoid common causes of failure—is to use the right metrics at each stage of their DevOps journey.

    There are a number of well-recognized metrics for measuring DevOps success, including:

    • Deployment time
    • Change failure rate
    • Release cadence
    • Lead time
    • Recovery time

    These metrics serve two purposes. They’re well-documented and benchmarked indicators of performance that can help teams focus on their biggest problem areas. And they’re measurable outcomes that prove the business benefits of DevOps, such as agility, improved collaboration, higher quality systems and resilience, not to mention significant time and cost savings and efficiency.

    Choosing the Right Metrics

    Teams must choose the right metrics at the right time to identify their opportunities for improvement and to measure the impact of each stage of DevOps adoption. The right metric empowers a team to solve their most pressing challenges, and to demonstrate the value of their work to their leadership.

    Here’s how the benefits of each stage of DevOps maturity should be measured: 

    Stage of DevOps maturity

    Most relevant metric(s)

    Choosing your release tools

    Deployment time

    Adopting version control

    Change failure rate

    Recovery time

    Automating testing

    Change failure rate

    Automating deployments (CI/CD)

    Release cadence

    Lead time

    Integrating backup into DevOps tools and process

    Recovery time

    Let’s expand on each stage and the appropriate KPI.

    Choosing Your Release Tools

    This is more relevant to low-code platforms, but still broadly applies to the wider tech world. Even in the most mature of development processes, there will be manual steps. Getting changes into version control, reviewing changes, resolving conflicts, triggering releases—each of these is a manual step, and the effort involved in each of these manual steps is heavily dictated by the tools you use.

    Finding the right tool for each stage of your development and release process can therefore have a massive impact. The right deployment tool, for instance, can cut down deployment time by as much as 90%. This step alone increases efficiency and ultimately saves companies vast amounts of time and money. In extreme examples, teams that save so much time by picking up a couple of new tools to patch up their manual processes can be tempted to stop there. This is why teams need to look at other metrics as they proceed to adopt more DevOps processes.

    Adopting Version Control

    For most teams, a world without version control is unthinkable, having built their development and release processes around it a decade or more ago. For others, particularly in the low-code space where it’s yet to reach ubiquity, it can feel like an unnecessary burden. The introduction of version control is an extra step in the releases process and another level of complexity, with new tools and practices to learn. But version control is the foundation of an automated DevOps workflow. This step allows for more team collaboration and visibility into that collaboration, with code reviews and a detailed change history. Version control also allows teams to deal with merge conflicts early, rather than in the middle of a release. 

    After adopting version control, teams will see a reduction in their change failure rate due to code reviews and their recovery times will be shorter as version control offers rollback and a quasi-backup of the code and config captured by their version control system. But the real benefits of version control are in the DevOps processes it unlocks. 

    Automating Testing  

    It goes without saying that the purpose of testing is to improve the quality of releases. It does this in a few ways:

    • Unit testing – Exercising individual methods and functions in isolation generally results in more maintainable code, which has positive benefits for developer productivity and even things like handling merge conflicts.
    • Static code analysis – While this is not testing in the strictest sense, automatically verifying that code adheres to a set of best practices similarly improves code maintainability.
    • Integration testing – This type of testing ensures that combinations of classes interact and generate the expected results when combined together. This brings another level of validation that changes being made are likely correct.
    • End-to-end testing – UI testing is an example of this; exercising the system in the way a consumer would and verifying that the results are deterministic and correct. This also includes regression testing for key workflows and sometimes even acceptance testing.

    The more of these testing approaches a team uses, the more confidence they’ll have in their changes and the less human verification and intervention will be required. As teams’ DevOps processes mature, it’s often the human element that becomes the biggest bottleneck, so confidence in deploying changes is very valuable.

    When teams introduce more testing and test automation to their release pipeline, the right metric to watch is change failure rate—the proportion of releases that include a bug or error. The immediate benefit of higher-quality code translates to savings in the long term as less time is spent finding and fixing bugs or reworking code.

    Automating Deployments (CI/CD)

    If choosing the right deployment tool significantly reduces deployment times, automating deployments with a CI/CD workflow eliminates the time it would usually take to run those deployments manually. However, deployment times aren’t the key metric for measuring the success of CI/CD implementation. The most important metrics to pay attention to here are release cadence and lead times, which are dramatically improved thanks to painless, automated deployments. 

    With automated deployments, teams can increase their release cadence; rather than releasing huge batches of changes once a month, they might begin to release smaller packages every week. Ultimately, the goal is to have a steady stream of incremental improvements being released on-demand, and usually multiple times per day. Accelerated release cadences then result in reduced lead times—the period between work being finished and finally released to production. This is beneficial for developers and their end users as more frequent releases create more opportunities to act on user feedback earlier in the process. 

    Integrating Backup Into DevOps Tools and Process

    Recovery times are a key metric for DevOps success, as robust DevOps processes will improve the speed at which teams can restore after an incident. For even better resilience and faster recovery times, teams should integrate backup into their DevOps tools and processes.

    Restoring from a backup is comparable to a deployment, and this means that restore processes also can harness the power of DevOps so they feel like a familiar process—not something unusual that teams struggle to execute at the point when it matters most. It’s also important to make it as easy as possible to back up on demand just before a risky release.

    Downtime and data loss are massive costs to any business, so improving recovery times is a huge benefit of a fully mature DevOps process.

    See the Benefits of Maturing your DevOps Process

    When teams pay attention to the right metrics, they understand the benefit of each DevOps process individually and they are motivated to keep maturing their DevOps processes as a whole. It’s all well and good to take the pain out of deployments and releases, but high-performing DevOps teams effectively make deployments an asset for releasing more often and getting feedback earlier. As a result, they are better equipped to deliver more value to their companies—sooner.

  • JFrog Acquires Upswift to Push DevOps to the Edge

    JFrog Acquires Upswift to Push DevOps to the Edge

    JFrog today revealed it has acquired Upswift as part of an effort to extend its DevOps reach to edge computing platforms. Terms of the acquisition were not disclosed.

    Upswift has created agent software that is deployed as a binary file on an edge computing platform that makes it possible to continuously deliver software across a fleet of edge computing devices and platforms. That approach makes it possible for organizations to deploy software regardless of whether the target platform is only running Linux, containers or a virtual machine.

    JFrog CEO Shlomi Ben Haim said the agent software developed by Upswift will be integrated with the continuous integration/continuous delivery (CI/CD) platform created by JFrog to make it easier for organizations to manage the full life cycle of application workloads deployed on edge computing platforms.

    JFrog Upswift

    Organizations that deploy software on edge computing platforms and devices today typically have massive backlogs of updates that they are unable to deploy in a timely manner, said Ben Haim. By connecting the Upswift agent software to a (CI/CD) platform designed from the ground up to support binaries, it becomes possible to reduce that backlog as part of what JFrog describes as its ‘liquid vision’ for deploying software, he noted.

    In addition, after acquiring Vdoo to gain access to a set of analytics tools that discover vulnerabilities in application binaries, JFrog will also be able to bring DevSecOps best practices to edge computing applications as well, noted Ben Haim.

    The number of application workloads being deployed on edge computing platforms is rising sharply as organizations seek to process and analyze data closer to the point where it is being created and consumed. In fact, that shift is at the core of digital business transformation initiatives that depend on edge computing platforms and devices running, for example, internet of things (IoT) applications to process data in near-real-time.

    Upswift CEO Amit Ezer said it has already become apparent that the existing processes of updating software on edge computing platforms and devices are capable of handling that influx of software. Upswift created its tool to enable developers to assume more control over those processes, as responsibility for managing and securing software continues to shift left, he added. The issue in many cases is that organizations lack the ability to consistently deploy software on edge computing platforms, which results in a hodgepodge of homegrown tools being created to deploy software on edge computing platforms using largely manual processes.

    It’s too early to say just how much software will be deployed on the edge, but there may come a day when there is more software at the edge than there is in the cloud. Of course, the real challenge will be unifying the management of DevOps processes, regardless of where any given piece of software might be deployed. In fact, as edge computing continues to evolve, the line between edge computing and cloud computing will continue to blur until they are little more than natural extensions of each other.

  • 3 Pillars of Autonomous IT Ops

    3 Pillars of Autonomous IT Ops

    Imagine a future where human IT operators and intelligent systems work together in a virtual operations room; the systems do the heavy lifting while the humans supervise. This may seem like science fiction, but the truth is we are already on the way to that future with AIOps. While actual AI-augmented teams are still quite a way off, AI/ML tools are slowly but surely becoming more ubiquitous within IT Ops, NOC, DevOps and SRE teams and leading us into this not-too-distant future.

    In the last five years, the number of internet users grew by 50%. In the last two years alone, more data was generated than in the entirety of human history up to that point. Today, 5 quintillion (that’s 18 zeros) bytes of data are produced by smart devices daily. These massive volumes of data are having a huge impact on digital operations.

    Looking at the enterprise, we see similar trends: More tools generating more data and with it, greater complexity. This is due to many factors, including the transition to the cloud, M&A activity that introduces new tools and data into an existing organization and the adoption of new technologies and methodologies such as CI/CD, microservices and more. At some point, there is so much data and everything becomes so complex that IT Ops teams reach a breaking point where they can no longer manually manage the data being generated.

    And when that point is reached, outages and service disruption become an everyday reality, as IT teams are pulled away from their strategic, business-critical projects into what is essentially firefighting. 

    Implementing more observability and monitoring tools simply exacerbates the problem, and scaling teams isn’t cost-effective in the long term. The more forward-thinking solution is automation, and even more so, autonomy. 

    The Road to IT Ops Autonomy 

    Autonomy is the ability of IT Ops to operate with minimal to no human intervention. It relies on the automation of the incident management life cycle so that humans are needed only for supervision, in the same way that a single operator can supervise a modern and highly complex robotic car assembly line.

    How can we hope to reach IT Ops autonomy?  The following three proposed pillars can act as a guide: 

    Pillar One: Automate

    Pillar Two: Predict and prevent 

    Pillar Three: Democratize

     Pillar One: Automate

    We must automate as much as possible across every step in every aspect of incident management. It won’t happen overnight, but with time and with the help of AI/ML it is doable.

    Consider the autonomous driving analogy.

    Just a few years ago, cars were manual, with little to no automation. Then we moved through partial autonomy to where we are today–conditional autonomy–where the car can drive itself in ideal conditions. This is a level of automation that we are beginning to accept; some of us cautiously, others with more enthusiasm! Meanwhile, progress continues toward higher levels of autonomy, in which vehicles will be able to drive themselves in most conditions. One day, maybe, we’ll get to the holy grail of full autonomy, where the car will drive on its own–perhaps we won’t even need a steering wheel!

     

    Similarly, in IT Ops, automation is a process and it will take time to build not only the capabilities required but also the necessary trust in AI/ML technology. Such a journey typically begins with human-driven automation, as “if-then-else” types of automation are added manually. For example, if a certain alert appears, a ticket will automatically be assigned to a specific person. As our confidence grows, the next stage could be AI-driven suggestions, whereby AI applies what it has learned from previous human actions to suggest what to automate, but the human expert decides whether to accept the suggestion or not. 

    At some point, we will progress to allowing AI to partially automate some tasks—perhaps those that are less risky, such as collecting diagnostic information and adding it to the alert, so that the incident responder automatically receives both together. We may even automate some parts of our system; perhaps within a staging environment that isn’t production-critical and, as such, is a good place to validate that the automation is working as expected. Over time, as our AI and ML capabilities develop and our trust in them grows, we may eventually reach full AI-driven automation, with no human involvement at all—the AI will simply detect the fault and resolve it. 

    Pillar Two: Predict and Prevent 

    Typically, when assigned an incident, a responder will take note of the incident’s priority. But they will also rely on their common sense and past experience in deciding how to respond. So, for example, if a P3 incident had recently escalated to a P1 in 30 minutes and a similar P3 incident now appears again, our responder will probably prioritize it, even over other incidents, to avoid a similar escalation.

    While this is one of the advantages of human logic, the flip side is that the IT environments in which we work are vast and complex—no human can possibly remember every potentially relevant detail. What’s more, if a different responder dealt with the earlier incident in our example, the current responder would not have the same insight. What is needed is an element of enrichment that can input leading indicators into the platform for the next operator to see and act upon to prevent an incident.

    This is where AI can help. 

    By identifying patterns in the data haystack—linking different incidents to the same root cause, for example—AI not only generates insights that help resolve these incidents faster but can also predict a certain outcome. This is done by anticipating a future chain of events so that appropriate preventative action can be taken before the event occurs. For now, this preventative action may be manually taken by the operator, but at some point in the future automation may enable the problem to be identified and fixed by the AI with no human intervention before it results in an outage.

    Pillar Three: Democratize

    The last pillar that will help us achieve full autonomy in IT Ops is democratization of AI. AIOps should not just be accessible to advanced, mature organizations with an army of data scientists working behind the scenes. Just as anyone can accomplish almost anything online these days with just a few clicks, any organization should be able to adopt AIOps and enjoy its benefits! 

    Democratization is about making relevant information accessible to anyone who needs it, when they need it, and in a manner that is easily applicable and actionable. AI should be accessible through simple user interfaces, with no need for coding; the benefits of machine learning should be available to people that don’t (need to) understand machine learning models. 

    This democratization can be achieved by making AIOps platforms simple and accessible to all—from the admin to the user. For example, an AIOps platform might automatically suggest how admins should prioritize incidents for operators, to make triage easier and provide instructions that even a non-expert can follow. The more such features are built into AIOps platforms, the more democratized AIOps will become.

    Data volumes will continue to grow exponentially in the coming years, and the only way to eventually conquer the tyranny of overwhelming data will be to reach autonomy in IT Ops. 

    If you’re an AIOps vendor, keep the three pillars in mind so your customers can continue to progress towards autonomy.  If you’re an IT Ops leader or tooling architect, and IT Ops autonomy is part of your vision, make sure that your AIOps vendors’ platforms and tools have been architected with these pillars and their principles in mind. 

    How far are we from achieving autonomy? Only time will tell. But one thing is for sure—we will get there, and if the COVID-19 pandemic has taught us anything it’s that we may be seeing machine-augmented teams sooner rather than later. 

  • 3 Building Blocks for Enterprise Digital Transformation

    3 Building Blocks for Enterprise Digital Transformation

    DevOps has gone from a new idea to a buzzword in every boardroom over the last decade. With society increasing its reliance on digitization and pushing their preferences for speed and convenience, enterprises of all types have had to change their business culture to help their teams to work together with greater speed and autonomy.

    While the pandemic has undoubtedly heightened the importance of DevOps in the workplace, its widespread use predates the last few years. HGS Digital research from as far back as 2018 found that only 3% of software development businesses had never heard of DevOps. The 2021: Future of the Atlassian Ecosystem report revealed that 54% of organizations have implemented a DevOps strategy—up from 48% in 2020. A further 27% of respondents are hoping to adopt a DevOps strategy in the next three years.

    Indeed, DevOps is now mainstream, and the uptick in job listings for ‘DevOps engineers’—as well as enterprises’ quickness to tout their ‘experienced DevOps teams’—is perhaps further evidence of this. However, some of these postings and sentiments can also be considered red flags as they suggest that organizations are applying DevOps practices to isolated teams. Or worse, they are simply labeling traditional engineering roles with the DevOps label rather than implementing a complete mindset shift that applies to the entire organization and enables it to release reliable products at a much faster pace.

    With this in mind, here are three key building blocks for enterprises looking to avoid common downfalls and effectively scale their DevOps strategies. These will help them change the way they work and think across all aspects of development and operations.

    Changing Culture and Empowering Employees

    Everyone needs to be on board for DevOps to be successful, and buy-in is strongly dependent on the business culture they are working within. For example, Gartner predicts that through 2022, 75% of DevOps initiatives will fail to meet expectations. And the reasons why have to do with issues around organizational learning and change.

    To put it another way, if employees don’t understand DevOps’ deeper thinking and benefits both to them personally and to the wider group, they will naturally be resistant to the change. It is, after all, an entirely new way of thinking for many departments, especially operations teams who traditionally reduce risk by minimizing change and taking things slowly. DevOps encourages them to speed up with minor changes delivered through continuous integration operating in a fail-fast and often fail-forward pipeline.

    Therefore, educating employees is vital for success. In particular, organizations should give staff a clear idea of how it will feel to work differently. Shared definitions of success, shared vocabulary and shared visibility of the work to be done is key. With that—and along with sharing actionable insights that teams can apply to their work and then observe the results—no matter where they are in the pipeline, the real value of DevOps can be achieved.

    The reality is that people are the most critical component of any DevOps transformation, making their commitment to the cause non-negotiable for success. Despite the tech-heavy nature of software development, soft skills like communication, collaboration, flexibility and decision-making must be prioritized as much as security, coding and infrastructure knowledge. It’s imperative that all businesses look to adopt a DevOps mindset and implement these traits into their culture in the future.

    Applying Four Fundamental Processes to the Foundation of DevOps

    That said, a successful DevOps transformation can’t happen unless enterprises step away from traditional IT’s ‘trust but verify’ methodology. Instead, they must incorporate a set of essential processes that empower everyone to work more efficiently and effectively toward achieving faster, high-quality software delivery to suit the end-user’s needs. These fundamental processes include collaboration, automation, continuous integration and continuous delivery (CI/CD) and monitoring.

    While collaboration has been one of the most significant rallying cries for enabling globally distributed IT teams to align their goals and meet their objectives, automation has also been highly crucial. Many of those ingrained in DevOps environments will tell you that automation is what makes DevOps highly valuable. It helps teams achieve high levels of efficiency and allows employees to focus more on complex problems and less on repetitive tasks. When a company’s automation strategy is aligned with its broader business goals, it can help deliver value at scale.

    But as is the nature of software development, problems can arise. And this is where continuous integration and delivery come into play; it forces developers to merge their source code updates with others far more frequently than they might otherwise choose to. Operations teams can ensure they’re not creating a bottleneck with manual testing to get releases out to users. The approach means smaller changesets; helping to identify, target and resolve bugs more efficiently. This increases the flow of work, the quality of releases and helps to avoid nasty problems at the end of the development cycle.

    But there’s no point in having a speedy, heavily automated CI/CD process with a genuinely collaborative culture if enterprises don’t take the time to measure what’s working and assess which actions need to be improved. This is why an effective DevOps plan needs to be built on the mantra ‘implement, measure and improve’. After all, DevOps is about maximizing value and efficiency through increased visibility of work and constant feedback. Measuring real-time metrics becomes vital to ensure business’ transformations are in line with their objectives at all times. If enterprises stick to these fundamentals, they’re well on their way to a successful DevOps strategy with complete buy-in.

    Adopting the Right Tools for Transformation

    As teams grow and DevOps processes scale, predictable performance is imperative. Therefore, the enterprise’s tools must be available around the clock and should perform at a high level to support their collaborative pipeline.

    But using DevOps tools doesn’t automatically mean organizations are applying DevOps principles. The value of tooling is limited by businesses’ commitment to using them holistically with the sort of culture and processes mentioned above. It is this combination that allows them to transition how they develop and deliver their products. With all that said, it’s important to understand how important it is to adopt the proper toolchain—and the wide variety of choices and combinations available—this is why it’s easy to understand why decision-makers might become overwhelmed.

    After all, there’s a lot to keep in mind. They need to choose tools that step up their collaboration, keep context-switching to a minimum, aid automation, encourage visibility and allow for effective monitoring. But there’s also the risk that they’ll pick the solutions that they feel most comfortable with, which can be counterintuitive to transformation. This approach can lead to a tool binge, bringing on too much technology where integration isn’t a priority, and leaving enterprises with nothing that does the job they need.

    With this in mind, enterprises need to take the time to consult on which tools are right for them while constantly keeping the fundamentals of DevOps in mind. To illustrate just how valuable this is, DORA and Google Cloud’s 2019 State of DevOps report found that the highest performing engineers are 1.5 times more likely to leverage easy-to-use tools. Meanwhile, failure to do this is perhaps a primary reason why a separate study from the DevOps Institute uncovered that the DevOps transformation journey is still arduous for more than 50% of respondents.

    Taking the Next Step Toward DevOps

    A successful DevOps transformation rests on in-depth understanding, mass buy-in and an organization-wide adoption of fundamental principles and practices. When this agile mindset is applied to the people, processes and tools that make up an organization, enterprises can drive significant change across their teams while increasing their products’ reliability, stability and security.

    It’s always important to keep in mind who DevOps is designed to benefit the most—the customer. This mindset is particularly valuable in today’s landscape, where businesses and consumers alike are flush with more choice than ever. With speed and convenience now at the forefront of most people’s purchasing decisions, DevOps can play a huge role in helping organizations strive for greatness and offer the customer something they didn’t even know they needed. Using the above building blocks, organizations of all sizes can dramatically improve their customer experiences and work more efficiently to quickly get the right products into their hands.