devopsy.kinsta.cloud

Tag: DORA

  • Using DORA Metrics to Measure DevOps Success

    Using DORA Metrics to Measure DevOps Success

    The DevOps Research and Assessment (DORA) metrics have fundamentally changed how leaders interact with developers—and, in turn, how developers interact with their code. Although these metrics have changed significantly since their introduction in 2014, their intent remains to provide a numbers-based approach to improving the developer experience.

    DORA metrics play a critical role in many development workflows. These metrics posit that four focus areas—deployment frequency, lead time for changes, change failure rate, time to restore service and reliability—can differentiate high-performing software engineering teams (elites) from lower-performing teams (non-elites).

    Why is this a meaningful difference? High-performing teams reliably achieve faster time-to-market by removing menial tasks from their workload and expediting the development and deployment processes. Elite teams provide a better customer experience, deploy robust code faster and maintain better availability. And engineering teams that codify their procedures and track using DORA metrics are more likely to achieve these results in a shorter time frame.

    Why are DORA Metrics the Baseline for Many Developers?

    Most departments have tangible objectives and key results (OKRs) to present to executives. For example, the finance department typically looks to achieve a certain revenue threshold, while the marketing department may report on social media traffic or engagement.

    Software engineers have spent years grappling with how to demonstrate success to non-engineering departments. Although it’s relatively easy to identify when a deployment goes wrong, many developers have difficulty translating how these breakdowns occur. Engineering teams cannot generate buy-in for necessary process improvements or tool adoptions without the vernacular to discuss success benchmarks.

    DORA metrics provide a language for developers to use when speaking with executives and other teams. Moreover, DORA metrics enable developers to understand their process’ weak points and identify areas for improvement internally.

    Making the Most of DORA in 2023

    It’s incredibly frustrating to fall short of development goals or hit roadblocks like post-deployment errors and bugs. DORA metrics are the secret to circumventing these challenges. Here’s how DORA can go beyond codification to help your team achieve quicker, more reliable development and deployment:

    Accelerate Time to Market

    Tracking DORA metrics can significantly curb a team’s time to market—a key competitive advantage in the fast-paced world of software development. Enterprises that deploy more quickly and frequently accrue greater revenue over time than less efficient enterprises.

    A streamlined deployment process looks different from organization to organization. That is why DORA metrics are essential; they pinpoint each organization’s specific areas of improvement, enabling DevOps leaders to make informed decisions about remediation. Common indicators for lagging time to market include non-orchestrated deployment processes and manual testing procedures, which distract developers from what they do best: coding.

    Leaders looking to isolate issues with time to market should scrutinize their organization’s deployment frequency, as this metric can indicate process inefficiencies. Deploying irregularly? Your team may be batching too many updates, leading to overloaded, riskier deployments. Alternatively, infrequent updates may mean your organization has outgrown manual deployments. If that’s the case, consider adopting continuous integration and deployment (CI/CD) processes—these offerings improve time-to-market through immediate feedback and automated testing. Successful CI/CD implementations are proven to increase daily deployments by over 1000%.

    Improve Development Processes

    Although highly interconnected, deployment and development delays stem from different issues. Think of it this way: Low-performing engineering teams might deploy frequently—several times a day, even—but without quality assurance, those deployments are likely erroneous and may lead to extended system downtime.

    DORA metrics are a powerful way to isolate deployment issues from gaps in the development process. For example, an engineering team that tracks DORA may find their deployment frequency optimal, but their time to restore service and change failure rate lagging. This discrepancy indicates the team needs to improve its testing practices by rigorously reviewing code and managing application dependencies better. Additionally, the team should consider implementing more advantageous deployment and rollback processes, including but not limited to progressive deployments and automated rollbacks.

    Remove Menial Tasks From the Workload

    Some organizations approach DORA metrics and other OKRs as productivity benchmarks that can be wielded against individual engineers. This perspective has even tinged how many developers view DORA metrics. Yet Google research demonstrates DORA metrics are most successful when adopted by high-trust, low-blame cultures. In these enterprises, DORA metrics reveal organizational opportunities for improvement—never should they reveal lagging individual performers or provide reasons for unnecessary busy work.

    In fact, DORA metrics often eradicate rote tasks from developers’ workloads. Teams that use these metrics to identify and implement better incident management processes spend less time fighting fires (e.g., reworking code after a costly manual rollback) and more time doing higher-value work, like building new experiences. Automated processes like automated rollbacks and CI/CD empower developers to improve the quality of individual deployments while also reducing stress

    Make DORA Work for Your Engineering Team

    DORA metrics aren’t going anywhere soon. While they continue to evolve, like the recent removal of the elite category, they remain the industry standard for engineering OKRs, they provide a robust framework for success and a common language for developers and non-engineering stakeholders to communicate about performance, challenges and improvements in the software development process. DORA eases the collaboration and decision-making processes, creating an environment where leaders can consistently and effectively make data-driven improvements.

    But these improvements are only possible when engineers are empowered by beneficial processes and tools. I encourage leaders who implement DORA metrics into their benchmarking to take results seriously. Is your organization’s time to market lagging? Interrogate your current developer workflow—and actively take steps to address issues through developer-first solutions. Doing so will not only improve your organization’s software delivery capabilities but also foster a culture of continuous learning and improvement.

  • DORA Report 2022: The Magnitude of Software Supply Chain Security  

    DORA Report 2022: The Magnitude of Software Supply Chain Security  

    The term ‘software supply chain security’ (SSC) can be interpreted in many ways. Following the White House executive order in May 2021 and the European Cyber Resilience Act (CRA) of 2022, both governments and corporations are taking a more active role in ensuring their software is secure. Fundamentals such as having a software bill of materials (SBOM) and being able to cross-reference it with known common vulnerabilities and exposures (CVEs) are now the standard, but supply chain security covers far more than just the software that builds your products.

    To create certainty and clarity, leading standardization bodies such as OWASP, OpenSSF and others have set up ongoing efforts to define and mitigate SSC threats. They have active working groups, SIGs and TACs all working in vendor-neutral ways to help define and solve parts of the SSC problem.

    A great example of this is the continuous work being done by Google Cloud’s DevOps Research and Assessment (DORA) initiative. More specifically, the subject was reflected in the September 2022 DORA report that focused on security in general and supply chain security in particular. Using the Supply-chain Levels for Secure Artifacts (SLSA) framework as well as the NIST’s Secure Software Development Framework (SSDF) as benchmarks for actively addressing supply chain security issues, the researchers assessed how organizations were successful in approaching and addressing threats at all levels of the organization.

    My view of the major findings of the DORA report are summarized below.

    Who Owns Software Supply Chain Security?

    The report stated that organizations should empower individual teams and developers to “own” security while administering a consistent security policy and posture across its products. While I agree with this, we need to examine both sides of the ‘ownership’ coin. By nature, this hybrid approach creates complexity—and even friction—between the security team that, by definition, is responsible for all facets of security and the developers that are being tasked to ‘fix’ security issues in addition to their daily tasks. There has to be common ground between developers, their tasks associated with security and the security team.

    Individuals in organizations that build software care about what they build; they have a stake in the overall success of the final product and this feeling of ownership strengthens that level of care. However, asking for such ownership while adding a security component to the mix without providing sufficient knowledge, tools, proofs, and “power” to make decisions could lead to  a lack of trust, confusion, execution problems and frustration. To alleviate this, the security team should provide developers the tools, education and resources they need to feel empowered to enhance security efforts at the source. Organizations that provide evidence in a scalable, digestible manner are successful in building trust and creating common ground between the developer, security and the security team.

    There’s also another dimension that could potentially create silos between developers and security teams rather than bringing the two together: There’s no one single security team or owner of SSC. SSC is tricky to define as a category; it involves shift left security, runtime security and everything in between. Some portions of it are being governed by the security office, some by AppSec, some by DevOps and some by the developer. The ownership of each isn’t well-defined and the entire effort is very difficult to consolidate as one process or methodology. In addition to upskilling the developer with tools and training, long-term education, thought leadership and access to good technology are the keys to gradually reduce these friction points..

    Security Processes Slow Down Development

    Defining the tools, methodologies and processes that should be employed in an organization to add sufficient security while not impacting development speed is not a trivial task.

    ­­The following are four principals I think correspond well with the SLSA framework and that help address the development slowdown challenge:

    Set preventive measures: Avoid using known vulnerable and malicious packages from the beginning of development. By eliminating vulnerabilities introduced in first-party code (due to bad coding practices) as well as misconfigurations, wrong encryption and secrets exposures, you are preventing known threats from entering your product.

    Examine proactively and continuously: The security state of software changes because the software itself always evolves and new vulnerabilities are continuously being introduced. It makes a software artifact that was just secured suddenly become susceptible to attack. This requires continuously rescanning and revisiting software artifacts as well as frequently recalculating any potential risk.

    Set preemptive measures: Create a plan that initiates a prompt response when a potential risk is discovered. Both development and security teams should create the plan for decision making and execution. The measures should include ways to identify vulnerable artifacts across development and production, how to understand exploitability and true impact and how to prioritize and remediate accordingly.

    Do it all in an automated manner: Create automated processes based on reliable data and scanning tools to dramatically reduce the effort needed by developers and DevOps teams.

    DORA report co-author Eric Maxwell said one of the main principles is to remain tool- and platform-agnostic while focusing on capabilities and practices. From that perspective, automation is essential and it is enabled by security-as-code. Having a common language–code–allows teams to communicate about security as well as collaborate. It drives the things we discussed earlier around democratizing the security process and helps break down silos.

    The DORA findings—alongside the topics described above—encourage community awareness and continued shared efforts to bring much-needed attention to software supply chain security. Gradually, SSC security will evolve, but as it does so, the only way to properly address it is by ensuring the developers and security teams are on the same page and both have ownership of responding to and resolving issues as they appear. For more in depth review of the DORA report I encourage you to view Leap Left for Security: The DORA Report Roundtable.

  • Three Trends That Will Transform DevOps in 2023

    Three Trends That Will Transform DevOps in 2023

    The onset of COVID-19 accelerated digital transformation-related initiatives like cloud migration, microservices, observability and CI/CD, giving long-delayed projects the budget and staffing they needed to build momentum. After almost three years of increased investment and accelerating progress, CTOs and software development teams are navigating how to enable the improvements in time-to-market and customer experience that they set out to achieve. In 2023, we’ll see enterprise DevOps mature and expand its business impact by placing a central focus on quality, delivering improved time-to-market, customer satisfaction and productivity.

    DevOps Matures as a DX Enabler

    The past three years have seen massive growth in cloud computing, observability and pipeline automation as enterprises raced to deliver new, high-quality digital experiences to consumers. Cloud spending alone increased from $270 billion in 2020 to 397.5 billion in 2022, according to Gartner. With new technologies in place, the focus shifts to processes that deliver business agility and efficiency.

    Under increased pressure to deliver faster, businesses need to revisit, evolve—and, in some cases, undo—decades of risk and change management practices built around brick-and-mortar experiences and long software release cycles.

    As digital engagement increases, so will expectations for software quality and scalability. We’ll need to invest in new tools and practices—from testing to observability and beyond—to ensure that we’re meeting or exceeding those expectations.

    The economic uncertainty will compel us to look for ways to improve productivity and decrease waste. Quality practices are central to both. We’ll continue to look for ways to shift left and catch defects early in our pipeline when they are the least expensive to fix. We’ll want to optimize our build, integration, testing and deployment processes to identify bottlenecks and causes of rework that drain the productivity of so many teams daily.

    DevOps, with its emphasis on automation and collaboration, will connect the dots between operational goals, technological investments and improved software. Established frameworks like DORA give software development organizations concrete ways to measure the efficiency and effectiveness of delivery pipelines. The key metrics outlined by DORA—deployment frequency, lead time for changes, change failure rate and time to restore service—enable teams to align on where they can improve development practices and quality practices.

    DevOps Will be Measured by Business Success

    The value of digital transformation and DevOps will also extend even further across the enterprise as CTOs evaluate DevOps practices with structured assessments such as DORA. With several years of accelerated digital transformation under our belts, the software development industry can build out a broader definition of DevOps performance over time. The DORA framework sets the stage for understanding how engineering practices impact the customer experience and, in turn, company growth. In one example, Liberty Mutual, a company prides itself on exemplary customer service, is already seeing how DevOps performance connects to business success.

    Architect Justin Robinson notes in their roundtable on data-driven decision-making and DORA metrics: “For organizations, this is improved organizational performance. If we want to keep investing in great tech and serving our customers as best as possible, [we need] behaviors and practices that we can adopt to continuously get better and hit on every front here. Happier engineers, a better culture and a better bottom line for our company.”

    In 2023, more enterprises will harness DORA to measure DevOps performance and its impact on company growth.

    Customers Will Reap DevOps Benefits

    We’re already seeing how metrics like deployment frequency can support an organization’s ability to attract and retain users. According to the 2022 Testing in DevOps Report, teams that increased deployments by 50%-100% were two-and-a-half times more likely to report high user satisfaction. The ability to innovate—without introducing defects to users—is the next frontier in DevOps success. Quality engineering will serve as a safeguard for DevOps teams so they can deliver new features faster and with confidence.

    The Testing in DevOps Report also illustrated the growing connection between digital transformation, DevOps and quality engineering. A full 88% of teams with highly automated pipelines and mature DevOps practices reported high test coverage, which in turn is connected to high customer satisfaction. Test coverage, a core metric for quality assurance teams, is becoming the connection between code and customers throughout development. Quality teams will expand their testing strategies to cover more aspects of the customer experience, including non-functional attributes like performance and accessibility. Automation, particularly automated testing embedded in CI/CD pipelines, will allow teams to adopt these additional quality factors without slowing deployments.

    Connecting Engineering Transformation to Business Success

    2023 is shaping up to be an unpredictable year in technology. But with the momentum propelling digital transformation to an enterprise-level priority, engineering leaders have the data and the tools to shift focus from DevOps processes to transformative outcomes. DevOps maturity, measured and improved with DORA, will help organizations extract more value from their digital transformations by supporting greater innovation and better customer experiences. Quality engineering, augmented with new testing strategies that measure more of the customer experience, will play a critical role in this evolution by enabling development teams to move faster with confidence.

  • Best of 2022: How DORA Metrics Can Measure and Improve Performance

    Best of 2022: How DORA Metrics Can Measure and Improve Performance

    As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    A technology company’s most valuable assets are its people and data, especially data about the organization itself. By knowing what data to track over time, engineering leaders can measure how efficiently their DevOps teams are operating and enable them to maximize their value stream to deliver the best possible product to end users.

    After years of research, Google’s DevOps Research and Assessment (DORA) team identified four key metrics for evaluating a team’s performance:

    1. Lead time for changes
    2. Deployment frequency
    3. Mean time to recovery
    4. Change failure rate

    DORA metrics have now become the standard for gauging the efficacy of software development teams and can provide crucial insights into areas for growth. These metrics are essential for organizations looking to modernize and those looking to gain an edge against competitors. Below, we’ll dive into each metric and discuss what they can reveal about development teams.

    Lead Time for Changes

    Lead time for changes (LTC) is the time between a commit and production. LTC indicates how agile a team is—it not only tells you how long it takes to implement changes but how responsive the team is to the ever-evolving demands and needs of users. The DORA team identified these benchmarks for performance in their Accelerate State of DevOps 2021 report:

    Elite performers: Less than one hour

    High performers: One day to one week

    Medium performers: One month to six months

    Low performers: More than six months

    LTC can reveal symptoms of poor DevOps practices: If it takes teams weeks or months to release code into production, there are inefficiencies in their process. One can minimize their LTC through continuous integration and continuous delivery (CI/CD). Encourage testers and developers to work closely together so everyone has a comprehensive understanding of the software. Consider building automated tests to save more time and improve the CI/CD pipeline.

    Because there are several phases between the initiation and deployment of a change, it’s wise to define each step of the process and track how long each takes. Examine the cycle time for a thorough picture of how the team functions and further insight into where they can save time.

    One should be careful not to let the quality of their software delivery suffer in a quest for faster changes. While a low LTC may indicate that a team is efficient, if they can’t support the changes they’re implementing or they’re moving at an unsustainable pace, they risk sacrificing the user experience. Rather than compare the team’s Lead Time for Changes to other teams’ or organizations’ LTC, one should evaluate this metric over time and consider it an indication of growth (or stagnancy).

    Deployment Frequency

    Deployment frequency (DF) is how often you ship changes; how consistent your software delivery is. This metric is beneficial when determining whether a team is meeting goals for continuous delivery. According to the DORA team, these are the benchmarks for Deployment Frequency:

    Elite performers: Multiple times a day

    High performers: Once a week to once a month

    Medium performers: Once a month to once every six months

    Low performers: Less than once every six months

    The best way to enhance DF is to ship a bunch of small changes, which has a few upsides. If deployment frequency is high, it might reveal bottlenecks in the development process or indicate that projects are too complex. Shipping often means the team is constantly perfecting their service and, if there is a problem with the code, it’s easier to find and remedy the issue.

    If a team is large, this may not be a feasible option. Instead, one may consider building release trains and shipping at regular intervals. This approach will allow the team to deploy more often without overwhelming your team members.

    Mean Time to Recovery

    Mean time to recovery (MTTR) is the average amount of time it takes your team to restore service when there’s a disruption like an outage. This metric offers a look into the stability of your software, as well as the agility of your team in the face of a challenge. These are the benchmarks identified in the State of DevOps report:

    Elite performers: Less than one hour

    High performers: Less than one day

    Medium performers: One day to one week

    Low performers: Over six months

    To minimize the impact of degraded service on your value stream, there should be as little downtime as possible. If it’s taking your team more than a day to restore services, you should consider utilizing feature flags so you can quickly disable a change without causing too much disruption. If you ship in small batches, it should also be easier to discover and resolve problems.

    Although mean time to discover (MTTD) is different from mean time to recovery, the amount of time it takes your team to detect an issue will impact your MTTR—the faster your team can spot an issue, the more quickly service can be restored.

    As with lead time for changes, you don’t want to implement sudden changes at the expense of a quality solution. Rather than deploy a quick fix, make sure that the change you’re shipping is durable and comprehensive. You should track MTTR over time to see how your team is improving and aim for steady, stable growth.

    Change Failure Rate

    Change failure rate (CFR) is the percentage of releases that result in downtime, degraded service or rollbacks, which can tell you how effective a team is at implementing changes. As you can see, there is not much distinction between performance benchmarks for CFR:

    Elite performers: 0-15%

    High, medium and low Performers: 16-30%

    Change Failure Rate is a particularly valuable metric because it can prevent a team from being misled by the total number of failures they encounter. Teams who aren’t implementing many changes will see fewer failures, but that doesn’t necessarily mean they’re more successful with the changes they do deploy. Those following CI/CD practices may see a higher number of failures, but if CFR is low, these teams will have an edge because of the speed of their deployments and their overall success rate.

    This rate can also have significant implications for the value stream: It can indicate how much time is spent remedying problems instead of developing new projects. Because high, medium and low performers all fall within the same range, it’s best to set goals based on the team and the particular business rather than compare to other organizations.

    Putting it All Together With DORA Metrics

    As with any data, DORA metrics need context, and one should consider the story that all four of these metrics tell together. Lead time for changes and deployment frequency provide insight into the velocity of a team and how quickly they respond to the ever-changing needs of users. On the other hand, mean time to recovery and change failure rate indicate the stability of a service and how responsive the team is to service outages or failures.

    By comparing all four key metrics, one can evaluate how well their organization balances speed and stability. If the LTC is within a week with weekly deployments but the change failure rate is high, then teams may be rushing out changes before they’re ready, or they may not be able to support the changes they’re deploying. If they are deploying once a month, on the other hand, and their MTTR and CFR are high, then the team may be spending more time correcting code than improving the product.

    Because DORA metrics provide a high-level view of a team’s performance, they can be beneficial for organizations trying to modernize—DORA metrics can help identify exactly where and how to improve. Over time, teams can measure where they have grown and which areas have stagnated.

    Those who fall into the elite categories can leverage DORA metrics to continue improving services and gain an edge over competitors. As the State of DevOps report reveals, the group of elite performers is rapidly growing (from 7% in 2018 to 26% in 2021), so DORA metrics can provide valuable insights for this group.

    But remember that data will only get you so far. To get the most out of DORA metrics, engineering leads must know their organization and teams and harness that knowledge to guide their goals and determine how to effectively invest their resources.

  • Culture a Stumbling Block to DevOps, DevSecOps

    Culture a Stumbling Block to DevOps, DevSecOps

    Organizations are struggling to improve DevSecOps practices as they confront a lack of cultural alignment and investment, according to a global Progress survey of more than 600 IT, security, application development and DevOps decision-makers.

    Many companies are behind in achieving their DevOps and DevSecOps goals, with 73% of organizations saying they could be doing more.

    More than three-quarters (76%) of respondents acknowledged they need to be more strategic about how they manage DevSecOps, and 17% said they still consider themselves at an exploratory and proof-of-concept stage.

    Prashanth Nanjundappa, vice president of product management at Progress, said the most surprising finding was the evident contradiction between what organizations think is very important and their actual alignment, commitment or investment. 

    “While security is the number-one driver behind most DevOps and DevSecOps implementations, just 30% said they felt confident in the level of collaboration between security and development,” he added. 

    Meanwhile, 86% of respondents are experiencing challenges in their current approaches to security and 51% admitted that they don’t fully understand how security fits into DevSecOps.

    Nanjundappa pointed out that company culture is, in fact, biggest barrier to DevOps and DevSecOps success. 

    “We found 71% of respondents agreed that culture is the biggest barrier to DevSecOps progress, yet only 16% were prioritizing culture as an area to optimize in the next 12 to 18 months,” he explained. 

    The study revealed the top business factors driving the adoption and evolution of DevOps inside their organizations included a focus on agility as well as reducing the business risk of quality, security and downtime or performance issues.

    Organizations also recognized the need to implement DevOps to support a cloud mandate or their move to the cloud.

    In addition, concerns about public cloud workloads, increasing regulatory requirements and concerns about financial impacts are driving how security around DevOps is approached. 

    “Organizations are juggling lots of priorities in how they manage security with key focus areas for security being digital marketing efforts, employee apps and customer-facing apps,” Nanjundappa said. 

    Challenges with security include prioritizing the development of external-facing capabilities over internal apps, the ability to secure different workloads using different development and delivery methods, meeting delivery deadlines and difficulties meeting audit requirements.

    “Through the survey, we have seen some of the differences that stand out between organizations that have achieved higher maturity versus others,” he noted.

    Some are related to cultural aspects and bring to the fore a focus on business agility through fast and frequent delivery of application capabilities, as well as using DevOps to better manage distributed and/or remote work environments.

    Nanjundappa said among the best practices to ensure fruitful collaboration are stakeholder sponsorship on leadership for tools and outcomes aligned with the strategic direction of organizational goals.

    “Let teams innovate at the grassroots level with regard to tools and processes to accomplish expected outcomes,” he explained. “Identify teams which have seen success, celebrate those successes and implement those processes across other teams in an iterative model.” 

    As organizations head into 2023 with a tight IT labor market persisting, Nanjundappa pointed out a key concern will be to find the talent they need to make these DevSecOps initiatives a reality.

    “While it is ideal to hire pre-trained talent which fits their organizational needs, it will either be expensive or sometimes close to impossible for multiple reasons,” he said. 

    He recommended using a multi-pronged approach to bridging talent gap, which includes an on-ramp for hiring experienced key talent and reskilling through cross-pollination of experiences from different teams, including security, dev and ops. 

    Nanjundappa also advised upskilling existing teams with modern technologies, replacing legacy platforms and tech with cloud and cloud-native technologies and augmenting that with the right level of tools and processes, including policy-as-code and CI/CD. 

  • How a Mature DevOps Practice Can Strengthen Competitive Advantage

    How a Mature DevOps Practice Can Strengthen Competitive Advantage

    Saman Pourkermani, practice lead, operations effectiveness, Inspirant Group, co-authored this article.

    Software development and delivery has become an integral part of every aspect of a business and is no longer simply a support function. Nowadays, most of us, including those in business operations, know the anxieties of having to wait for specific environment release dates to deploy code for urgent fixes or updates to Dev, QA, UAT or production, which slows down productivity. Or worse, an environment has gone down with no backup environment. We all know systems have to be in place to ensure quality and security, but what makes some companies more successful than others when it comes to their DevOps operations?

    Any organization that builds and delivers software applications and digital services can benefit fromDevOps. The main difference is the maturity of their DevOps culture and practices. Companies with mature DevOps practices and cultural philosophies are not only able to build and deliver applications faster than those with traditional DevOps practices, but also learn and innovate faster.  Mature DevOps capabilities, tooling, and cultural philosophies enable faster innovation by automating and streamlining development, testing, deployment, and infrastructure management processes. The ability to outpace others from both a delivery and learning standpoint is a huge competitive advantage.

    Most organizations understand this and as a result, have been heavily investing in maturing their DevOps capabilities and practices. However, most struggle to make meaningful progress to catch up to leading companies.

    Why is That?

    The most obvious reasons are resource constraints–budgetary, human and/or systems. Many companies have fixed budgets and must balance the tradeoffs between setting up an expensive infrastructure (equipped with yearly upgrades) with other business needs for short-term/long-term benefit gains. 

    We believe a company’s DevOps maturity is directly associated with and significantly impacted by how well and disciplined they are at making business prioritization and tradeoff decisions, picking what they will focus/work on and what they will not – ideally leaning out how much work they commit to completing.

    It goes without saying that most companies that want to but don’t yet have mature DevOps practices have more work than they can handle, from setting up and documenting CI/CD processes to creating business continuity and disaster recovery plans to ensuring they have the right people for execution. These critical elements take time, effort, expertise, and budget to put together. While everyone wishes their teams could get it all done in a timely manner, the reality is that everyone has a finite amount of budget and capacity.

    Quite often, progress on DevOps capabilities and automation gets sidelined in favor of ongoing customer delivery work. The trick is to establish a mechanism for balancing teams’ capacity allocation so that it includes a healthy mix of work focused on becoming more effective and efficient at delivery (e.g., maturing DevOps capabilities, improving architecture and design, etc.) and work to fulfill customer needs. In this way, a percentage of a team’s capacity can always be devoted to advancing DevOps delivery capabilities while the rest can be used to address customer requests.

    How Can This be Accomplished? 

    Start With Culture. Culture is a critical factor when it comes to having a mature DevOps practice. Team members must fulfill their individual roles but also operate as a cohesive unit. The mindset of each team member must be focused on the greater good of the whole – not individuals. Has the team established a way of working together that fosters accountability along with encompassing growth and innovation?

    Focus on Process. Progress cannot be made without identifying a clear, sensible process. For example, having a well-documented and executed CI/CD process is paramount to DevOps maturity to address all developer pain points and ensure a smooth deployment process. The key is having a documented process that establishes the team’s way of working and addresses the following: Are the right team members engaged? Are there special circumstances that need to be accounted for during the development/configuration/deployment processes? Does everyone have the same understanding of the process at each stage?

    Automation and Tools. Have your processes been automated in a way that allows the DevOps practices to happen in the same way each time, without manual intervention? Do you have and know how to use the fundamental tools that all successful DevOps practices need (e.g., build and test code continuously, source control (e.g., GitLab), environment configuration management, performance measurement (e.g., load balancing))? Are deployment pain points adequately identified and addressed to prevent coding/release bottlenecks?

    Business Prioritization and Trade-off Decisions. We believe this is the most important aspect of becoming a mature DevOps practice because it allows for progress over time. Balancing the teams’ capacity to include advancing DevOps delivery capabilities is critical for establishing and sustaining a competitive advantage. Optimizing your DevOps capabilities allows you to free up and capitalize on the time, talent and performance of your team members for learning and innovation.

    As Jack Welch once stated, “The only sustainable competitive advantage is to learn faster than your competition and to be able to act on what you have learned.” When a software organization becomes too rigid without an adaptive DevOps edge, it will be left in the dust. 

  • Driving Organizational Success With the 2022 State of DevOps Report

    Driving Organizational Success With the 2022 State of DevOps Report

    “Ship it!”

    We have just completed the research and analysis for the 2022 Accelerate State of DevOps Report (SODR) and oh boy, do we have some interesting things to share! First, let me take a short step back for those of you that may be new to the DevOps Research and Assessment (DORA) and our annual report, the SODR. DORA is an academically rigorous research program that seeks to answer the questions, “What practices enable teams to be high-performing software teams? And how do these practices impact organizational performance?”

    Since 2014, we have surveyed 33,000 practitioners around the world spanning all major industries. The project is intentionally–and staunchly–tool- and platform-agnostic. The research builds on itself, and each year we seek to both evaluate previous findings and expand in other directions or areas of research. This is important to note because many of the core findings, such as “software delivery performance drives organizational performance,” have been validated year after year. I would encourage you to go back and read all the reports from years past.

    Security

    This year, we doubled down on our 2021 research into software supply chain security by looking at the technical practices that improve software supply chain security and the non-technical practices that impact an organization’s ability to excel at securing their software supply chains. We leveraged two frameworks to focus our research: Supply Chain Levels for Software Artifacts (SLSA) and NIST’s Secure Software Development Framework (SSDF). Below is a summary of some of our key findings in this area:

    • Shifting left on security is a broadly adopted practice. Our research shows that two-thirds of respondents are actively pursuing software supply chain security by seamlessly baking security into the development process.
    • Culture is the primary driver of security practice adoption. One might expect that technology is the primary driver, but our research showed that a generative organizational culture (eg. performance-oriented, highly cooperative, shared risk) leads to healthier software practices.
    • Technical practices around CI/CD predict success in security. Companies that use source control, continuous integration (CI) and continuous delivery (CD) have more established SLSA practices. These practices shift security left to the developers and ensure consistent security scanning.
    • Cloud enables secure software practices. The five characteristics of cloud computing as defined by NIST enable the successful adoption of software supply chain security which, in turn, predicts greater organizational performance.

    Our research showed that companies who prioritize and excel at securing the software supply chain experienced fewer service outages, anticipate fewer security breaches and demonstrate high levels of both software delivery performance and organizational performance. The data showed that through the use of modern practices like continuous integration, teams can improve their security posture and even amplify the positive impact these security practices have on software delivery metrics (MTTR, deployment frequency, lead time-to-restore service) and overall organizational performance.

    Culture

    We saw above that the biggest predictor of an organization’s application development security practices was the presence of a generative organizational culture based on shared risk and information sharing. We also see that elements of these types of cultures lead to higher overall organizational performance, as well. Our research showed that high organizational performance can be achieved by fostering environments that are:

    • Supportive: Teams that felt supported and that they had buy-in from leadership (e.g., more financial support, more allocation of resources, sponsorships, etc.) were associated with high-performing organizations.
    • Stable: Teams that did not experience much change in membership over the last 12 months were more likely to be a part of high-performing organizations.
    • Flexible: Organizations with higher levels of working flexibility around where work was performed–remote, in-person or hybrid–were higher-performing overall.

    We also looked at burnout again this year and expanded the scope to understand which elements of culture contributed to lower levels of burnout. We found that generative culture, team stability and work flexibility all contributed to a reduction in burnout among employees.

    Reliability

    In previous years, our research told us that those that excelled in technical practices also excelled at organizational performance. This year, we have more nuanced data about this topic. This year, we saw that software delivery does not predict strong business outcomes unless these practices were paired with reliability. Think about it–will a customer be satisfied with new features if the service isn’t stable? What is the benefit of pushing code quickly into a fragile environment? Reliability is an essential component of driving organizational performance through software delivery performance.

    We also see that site reliability engineering’s (SRE’s) impact on organizational performance is non-linear; reliability engineering practices often do not result in additional reliability or organizational performance until a certain maturity is reached. It’s important for teams to know this and approach their SRE practice as an investment. It will likely not be sparkly ponies and unicorns initially as you build the reliability muscle but, as you advance, high performance and success are likely.

    Cloud

    Public cloud use is up a whopping 36% over 2021, whereas companies reporting no cloud usage at all are down by 50%. The use of hybrid cloud is up by 25%. Unsurprisingly, the use of cloud computing was associated with higher organizational performance. In past years we saw that it wasn’t “using cloud” per se that led to organizational performance, rather it was the achievement of the five essential characteristics of cloud computing–on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service. This year, we saw that cloud computing enabled things like reliability, continuous delivery and improved supply chain security which drive organizational success.

    Over 50% of folks who took our survey reported that they leveraged multiple cloud providers. We asked respondents what benefits they realized from using multiple cloud providers. Here are the top three:

    1. Availability
    2. Leveraging the unique benefits of each provider
    3. Trust is spread across multiple providers

    Given the fact that reliability seemed to be the key to being a successful software delivery shop as well as a successful organization, it is not surprising to see availability listed by almost 63% of respondents as a benefit derived from using multiple clouds.

    Summary

    Our research continues to progress and we continue to dig deeper and deeper into the capabilities and practices that impact your business. We see broad adoption on the topics of security and reliability which are both heavily anchored in culture. Good culture leads to success. Period. There is no magic culture button and it takes work, but it is very achievable. Start by defining organization-wide or line-of-business-wide goals and then just start continuously improving. Don’t worry about creating a three-year plan for improvement; create a one-month plan and just get to work instead. At the end of that month, evaluate your learnings, refine your focus and get back to work. With a commitment to hard work and continuous improvement, your investments should start paying off and, as an added bonus, you’ll have a better culture.

    We hope you have enjoyed these juicy findings as much as we have and we encourage you to read the entire 2022 State of DevOps Report.

    Also, please join us at dora.community to continue the discussion about these findings and to share and discuss your experiences on your journey toward excellent software delivery and operations.

  • BMC Adds Support for DORA Metrics to Mainframe Tools Portfolio

    BMC Adds Support for DORA Metrics to Mainframe Tools Portfolio

    BMC this week announced it has added support for DevOps Research and Assessment (DORA) metrics within its portfolio of DevOps tools for mainframe environments.

    John McKenny, senior vice president and general manager for intelligent Z optimization and transformation at BMC, said the latest edition of BMC Compuware zAdviser now includes a DORA KPI Dashboard that aligns with the widely-adopted framework for measuring DevOps efficiency defined by an arm of Google.

    The DORA metrics track deployment frequency, failure rate, lead time for changes and mean-time-to-recovery (MTTR). Tracking those metrics is crucial for IT teams that have mainframes because they need to make sure applications continue to be deployed on the platform versus being usurped by another platform that is perceived to provide a faster alternative for builds and deployment, noted McKenny.

    BMC also added support for automated webhook notifications to its BMC Compuware Abend-AID offering. That includes diagnostic and root cause analysis to resolve defects in test and production faster. In addition, BMC provides a connector to the machine-identity management platform created by Venafi. That integration promises to eliminate certificate-related outages that can derail an application development initiative.

    In general, DORA metrics are being used to identify areas where DevOps teams can improve. It is up to each organization to determine to what degree those metrics will be used to coach teams versus enforce productivity mandates. However, given the overall shortage of available DevOps talent, few organizations are employing metrics to force DevOps teams to improve. In fact, there is no universal set of DevOps goals that every organization is trying to achieve. Rather, most organizations are trying to determine what set of DevOps best practices makes the most sense to implement given their own goals and internal culture. In the case of the mainframe, most organizations start with a few projects before they begin to employ DevOps more widely, noted McKenny.

    The metrics defined by DORA framework are, of course, only a subset of the total metrics that DevOps teams should track. BMC, for example, makes it possible to track tools usage to provide DevOps teams with insights into what tools may no longer be relevant or required, noted McKenny.

    Mainframe environments also have a long history of using waterfall-based approaches to building and deploying applications that many of them will continue to use alongside DevOps practices, depending on the type of application being built and deployed. In other instances, distributed applications built using DevOps practices are being integrated with mainframe applications that now need to be updated using the same workflow. The mainframe is no longer a platform that is managed in complete isolation from distributed computing environments.

    The major challenge is that many organizations using mainframes often overestimate how difficult it is to adopt DevOps  best practices, McKenny said. Regardless of the platform, there is no doubt the overall rate at which all applications are being built and deployed will continue to increase. The mainframe is proving to be no exception to that rule.

  • GitLab Adds Fourth DORA Metric API to CI/CD Platform

    GitLab Adds Fourth DORA Metric API to CI/CD Platform

    The latest update to GitLab’s namesake continuous integration/continuous delivery (CI/CD) platform has added support for the application programming interface (API) for measuring change failure rates. This addition supports the fourth metric as defined in the DevOps Research and Assessment (DORA) framework.

    Version 14.10 of GitLab also expands the GitLab Runner Operator for Kubernetes to any distribution of the open source platform in addition to making it possible to manually trigger incident responses when needed.

    Finally, a compliance report that surfaces every individual merge request violation has been added along with a tool for setting up streaming audit events.

    Brendan O’Leary, staff developer evangelist for GitLab, said that as DevOps processes continue to mature, DORA metrics make it simple for IT leaders to measure overall progress. While it’s not likely that any single set of DevOps practices will be universally implemented across an entire enterprise, O’Leary noted it’s important for DevOps teams to keep track of key performance indicators (KPIs) that enable teams to identify areas for ongoing improvement. For example, it’s not uncommon for technical debt to accumulate in the absence of metrics that track the progress of builds, he added.

    In general, a transition to microservices-based applications is increasing the need for CI/CD platforms that make it simpler to build them. Each of these applications is constructed using microservices that are typically built independently of one another. The challenge is that each microservice has dependencies on other microservices, so the need for DevOps workflows to optimally manage that development process is becoming more widespread, noted O’Leary.

    In some cases, organizations are deploying a CI/CD platform for the first time to build these applications. In other cases, they are looking to replace a CI/CD platform that was adopted to build legacy monolithic applications. Regardless of the type of application, there is a need to continuously refine and improve software engineering practices to increase developer productivity.

    It’s not clear to what degree developers are voting with their feet to work for one organization versus another based on the overall quality of developer experience. Organizations that do have a consistent approach to software engineering, however, make it easier for developers to spend more time writing code rather than managing the build process. The challenge is to make sure the DevOps platform itself does not become a bottleneck that hinders the rate at which developers can build code. It’s also not clear whether organizations are building applications faster than they can secure it and ensure overall quality. The only way to know for sure is, of course, to track metrics.

    Naturally, not every developer is a fan of metrics. Application development is still as much art as it is science. There is, however, a general appreciation that the amount of code that needs to be written far exceeds the available talent to write it; any effort to make software engineering more efficient benefits everyone involved. As such, given all the code that needs to be written, a software engineer is often today the best friend a developer can have.