devopsy.kinsta.cloud

Author: Aruna Ravichandran

  • Continuous Testing: Solving the Biggest DevOps Bottleneck

    Continuous Testing: Solving the Biggest DevOps Bottleneck

    It’s time to start implementing the practices of Continuous Testing

    Earlier this year, Computing Magazine conducted a fascinating study around the adoption and benefits of DevOps in its DevOps Review 2017 publication. The findings that stood out most fascinating to me where around quality assurance and testing.

    In this survey, the question was asked, “Where are the main holdups in the software production process?” Of all the self-proclaimed DevOps practitioners, a whopping 63 percent said the biggest delays occurring in the QA/Test phase. In fact, this cited bottleneck was more than 2 times greater than the next proclaimed bottleneck, which they felt was in planning (29 percent).  In this same survey, another question asked, “What tends to lose out more in DevOps environments?” While 43 percent said, “Nothing,” nearly 30 percent of these DevOps practitioners said that quality was being sacrificed. When asked which area they wanted to focus on improving in the next year, over 40 percent opted for quality.

    Why is it that quality has taken such a hit?  The study cited two issues: 1) Testing and QA is often siloed from the other parts of the organization, and 2) Testing tools haven’t matured much in the past five years.

    Testing must become a cross-functional operation across the software development lifecycle (SDLC) spectrum, one that involves all teams. Agile developers and testing teams don’t have weeks to test, but hours. Testing hasn’t kept up because it hasn’t applied the methods and concepts of agile development have not been applied to testing and QA processes. By removing the constraints that force testing to the end of a development cycle, developers and QA teams can achieve accurate, automated testing from day one. They can create an integrated pipeline, capable of taking an idea from design to deployment, at speed, without compromising quality.

    Open-source, cloud-based solutions can be the catalyst that brings the speed of agile to testing.  Here are a few examples.

    Better In-Sprint API Testing: Since APIs have become the major component of applications, organizations are shifting their approach from starting testing when the user interface is complete to shifting the focus of testing at the API level.  But the hard thing is finding enough time in a development sprint to test them sufficiently.

    Today’s applications are a collection of microservices and API calls. UI testing alone just won’t cut it. UI testing is very brittle, and whenever it changes, testing breaks. Testing needs to shift left into the development cycle—into sprints. But if you shift testing left to be a focus of the sprints, what kind of API testing can you really do? How comprehensive can they really be?  Creating those API tests has to be quick, and simple.

    Dev and Test teams must utilize automated tools that not only generate API tests, but create an exhaustive set of test cases from the API, then execute those tests. These tools also need to support the most popular open-source testing frameworks allowing developers to run functional API tests as performance tests without any proprietary conversion issues. And, vice versa, developers must be able to take open-source performance tests and be able to run them as functional API tests.

    Orchestrating Testing: A Continuous Delivery Pipeline Manager: “Faster” is becoming table stakes with automation.  The facts show that companies who have widespread agile adoption and are deploying more frequently with significantly reduced lead times. But, this too often leads to release failures, quality issues and disorder with multiple—if not hundreds—of applications rolling through the SDLC. Plus, things slow down significantly with time-consuming handoffs and rework in the release process. DevOps teams, who release features and fixes frequently while significantly reducing errors and manual processes, need toolchain-agnostic pipeline planning, orchestration and optimization solutions that enable them to eliminate spreadsheets, automate test processes and feedback, and continually improve the delivery of revenue-generating features to end-customers. New CD SaaS tools help make this orchestration process easy.

    Performance and Functional Testing Using Open Source Frameworks: More and more organizations are trying to move from legacy testing tools. It’s not so much that these don’t adequately test, but they were never designed to meet the needs of agile development approaches. Rather, they were designed for releases that occurred two to three times per year. But that won’t cut it anymore.

    And as developers begin to incorporate testing activities in the coding process, they need simple-to-use, open-source tools. Luckily, there are such tools like JMeter, Selenium, Gatling, Locust and others. There are even tools that help you convert from legacy platforms to open source technology with ease, and guides to transform existing functional tests created with other tools over to load tests.

    These are just a few examples of how continuous testing eliminates the testing hurdle. With continuous testing, organizations can:

    • Automatically convert user stories from requirements management tools like CA Agile Central or Atlassian JIRA into the minimum number of test cases required for complete code coverage, and being able to adapt automatically to changing business requirements.
    • Reduce test cycle prep time by requesting test data from within requirements.
    • Request, reserve, and create test data at any time, even from requirements phase.
    • Virtualize APIs for better testing, while managing API access to developers.
    • Institute easy API testing that can generate comprehensive test cases (positive, negative, edge case, and SQL injection) and execute those tests automatically.
    • Democratize testing with SaaS-based testing tools so that developers can test early and test often.
    • Speed testing by feeding test data directly and securely to virtual services.
    • Use open-source testing tools such as JMeter, Selenium, Jenkins, Jenkins, Appium, Taurus, Gatling, Locust and more.
    • Orchestrate the application delivery pipeline to promote releases from one environment to the next as soon as all the tests pass and enable faster feedback loops.

    This is testing unleashed, going hand in hand with agile development to accelerate quality into production.

    By adopting next-generation testing practices, DevOps teams have the ability to test early, often, automatically and continuously. To learn more about continuous testing and why it is critical to DevOps and Continuous Delivery initiatives, visit this Continuous Testing informational page.

    — Aruna Ravichandran

  • Using DevOps Metrics Effectively

    Using DevOps Metrics Effectively

    If you’re a DevOps leader, you have a very tough job. You have to radically transform processes and behaviors that have been ingrained in IT for decades. You have to adopt new technologies quickly while still delivering code quickly and cost-efficiently. And you have to do all this better than your peers at other companies. Otherwise, you’ll simply slow your company’s progress in the new digital marketplace while your better-performing competitors gain advantage.

    The good news is that the right metrics can help you gauge your success. Metrics can both tell you if you are traveling down the right path, or need to adjust, and help you prove the success of your DevOps initiatives to the rest of the business. But first you have to get them—and just as importantly, you have to use them.

    I wrote two previous blogs talking about how to get the metrics—one outlining what not to measure and another that delved into determining which metrics are right for your organization. In this blog, we will explore the latter part: how to use metrics to ensure ongoing DevOps success.

    Consider the Audience

    Measurements of any sort have two primary audiences: internal and external. The same metrics can be used and gathered for both, but the intent—and how they are communicated—is different.

    For internal audiences, you are measuring things to gain a better understanding of your own performance. It’s important to know what adjustments need to be made: where to double down and where to change course. Like a racing car pit crew monitoring hundreds of sensors on a vehicle, your DevOps team should be constantly looking at ways to extract the maximum performance and value from your efforts. Metrics will help you do this.

    External audiences are less concerned about the nuances and details and more concerned with the actual outcomes. Going back to the racing car analogy, external audiences care about lap times and top speeds; not about the operating temperature of the engine. It’s important to keep this difference in mind when you are communicating results to business leaders. Don’t bog them down with nuances and details; instead, show the results.

    Best Practices for Communication

    For both internal and external audiences, there are five best practices to consider when communicating metrics:

    1. Automation. Automation isn’t just important for efficiency and consistency in the software development life cycle, it is also useful for capturing and communicating DevOps metrics. Tools that automatically record and communicate your results will help you keep your finger on the pulse of your organization without getting bogged down in measuring and reporting.
    2. Evolution. The savviest DevOps leaders don’t marry themselves to a specific set of metrics. They intuitively understand that the metrics they use to evaluate DevOps progress will mature as the DevOps practice itself matures. The metrics you measure in the first six months may not be the ones you measure in year two.
    3. Action. This perhaps goes without saying, but metrics need to be communicated to both audiences listed above. It can be challenging communicating results when they are not what was expected, but the best way to communicate less-than-optimal metrics is to have an action plan to remediate any possible issues.
    4. Communication. As noted above, action and communication go hand in hand. Once you know what’s working and what’s not working, you should have a solid plan on next steps. When something’s working well, the plan should focus on how to replicate success; when something is not working as well, the actions should focus on future improvements.
    5. Self-examination. Metrics aren’t just indicators of what your team is doing right and doing wrong. They’re also useful diagnostics for leadership decisions and leadership style. So remember to take the responsibility and share the credit.

    When you marry the above best practices with the other measurement tips outlined in my previous blogs, you will not only be able to gather the right metrics, you’ll know what to do with them. Remember that, like DevOps itself, the practice of measuring DevOps success is something that can be started slow and expanded.

    Not every organization has the time or the resources to measure everything that should—or could—be measured. So, start slow, focus on the metrics that matter to your organization and use the best practices outlined above to guide you. For those who want to learn more about DevOps and DevOps metrics, my book, “DevOps for Digital Leaders,” explores these topics—and more—in a much more detail.

    Happy measuring!

    — Aruna Ravichandran

  • Metrics Only Matter When They Matter

    Metrics Only Matter When They Matter

    Many studies have shown the significant value that DevOps can bring to a software organization. However, organizations on their own DevOps journey need more than general industry studies or case studies. Any practitioner needs to be able to show the value—specific to their organization—that the changes to people, process and tooling made in the name of DevOps are having meaningful results.

    Measuring the success of a DevOps practice or project is critical to securing and maintaining support for similar projects in the future.

    In a previous blog, I talked about some of the pitfalls that come from focusing on the wrong DevOps metrics and outlined a few general guidelines around what makes a good metric. Here I’ll expand on that theme and discuss more specifically what types of metrics can help you show progress within your organization.

    To review, in the previous blog we looked at the characteristics of a good metric. Specifically, good metrics should be obtainable, reviewable, incorruptible and actionable. With those as general guidelines, it’s helpful to consider the three core components of successful DevOps implementation to help us determine where to look at measurements.

    DevOps is built around people, process and technology. But it doesn’t make sense to measure the components of DevOps; instead, we need to think about the goal, or outcome that each of these achieves. For example:

    • People changes are focused on improving culture, collaboration and sharing
    • Process changes are largely implemented with the goal of driving efficiency & effectiveness
    • Technology changes contribute to efficiency and effectiveness, but primarily drive quality and velocity

    Together, all three of these should lead to significant customer and business value.

    Notice how certain words above are in bold? That’s not an accident. These four key areas are the foundation of a successful DevOps practice—and therefore an excellent place to focus your measurement efforts. The figure below illustrates how these four areas fit together, and provides a few examples of the types of things that can be measure in each area.

    Of course, it’s important to consider your own organization when considering what metrics to focus on, as well as the guidelines mentioned previously. For example, in some organizations people-related metrics could be difficult to obtain on a regular basis. That’s not to say they’re not important—it’s been shown that DevOps can have a meaningful impact on employee satisfaction—but these metrics may not be the primary focus for some groups.

    DevOps has a wide range of benefits and I’ve seen organizations often have their own primary motivation, whether it is costs, velocity, competition or something else. It’s best to focus on metrics that are aligned to your organization’s primary objectives.

    Metrics matter only if what you’re measuring matters to your organization.

    If you are interested in learning more about DevOps metrics, the book “DevOps for Digital Leaders” explores this topic in more detail.

    In my next blog, I’ll delve into what to do with the metrics, once you’ve gathered and analyzed them. Because even the best metrics are only as good as what you—and your organization—can learn from them.

    — Aruna Ravichandran

  • Beware False DevOps Metrics

    Beware False DevOps Metrics

    Metrics are important for any manager seeking to continuously improve critical work processes and the resulting work-product.

    That’s why DevOps leaders need DevOps metrics. With the right ones, those leaders can guide their organization’s adoption of DevOps best practices—progressively optimizing staff productivity, business agility and customer experience.

    But, what are the right metrics for DevOps success? And, what are the wrong ones?

    These next few blogs will offer field-proven guidance on DevOps metrics. Much of the material in these upcoming blogs can be found in the book, “DevOps for Digital Leaders,” which I co-wrote with Kieran Taylor and Peter Waterhouse. And I’d love for you to comment below so we can all share lessons learned about how to effectively measure DevOps maturity—and, just as important, how not to measure it.

    Change Your Metrics, Change Your Culture

    DevOps leaders tend to struggle with metrics because DevOps entails changes that are both operational and cultural—and span different teams or functional areas, including development, test and operations. So inexperience may cause leaders to adopt new ones that are counterproductive—or, as is often the case, continue using traditional ones that don’t align well with the cross-functional nature of DevOps objectives.

    Three pitfalls have proven to be especially common in the transition to DevOps:

    • Vanity metrics. These appeal to the “hero” mentality of individual coders, rather than the collaborative process goals of cross-disciplinary DevOps and continuous delivery teams. They include total lines of code produced and total function points created. If incentives are linked to these metrics, developers can focus too much on volume at the expense of other essential activities such as refactoring and design simplification.
    • Those that promote interteam conflicts. Metrics that pit teams against each other undermine critical DevOps behaviors—such as code-sharing, mentoring and constructive peer reviews. So, while gamification may be useful in other areas of the business, Agile leaderboards that rank teams based on the prevention of extraneous deployments and changes can actually work against DevOps culture.
    • Traditional metrics. Many DevOps leaders persist in utilizing traditional/incumbent ones such as mean time between failures (MTBF) and the ratio of full-time equivalents (FTEs) to servers. But while these may have been useful in the past, they can work at cross-purposes with DevOps transformation. After all, with faster delivery of services, some failure is to be expected. And as IT improves its responsiveness to the business, it must look past costs alone to total economic impact.

    What Makes a DevOps Metric Useful?

    While future blogs will propose some specific metrics that DevOps leaders should find useful in their efforts to bring about cultural transformation, here are a few general guidelines to consider:

    They must be obtainable.

    A metric won’t do you any good if you can’t actually capture it. So you need DevOps tools that help you capture the metrics you need. You may also have to measure some attributes indirectly. For example, rather than measuring employee morale directly, it may make sense to use staff retention rates—which you can readily capture—as your performance indicator.

    They must be reviewable.

    Any metric you choose to employ as part of your DevOps leadership effort must be able to stand up to rigorous scrutiny in a business context. Your credibility as a leader depends on your ability to communicate its relevance to the stated values and mission of the team.

    They must be incorruptible.

    It’s never a good idea to promote metrics that can be influenced by individual team members or any group as a whole. You should also be on guard against metrics remaining entrenched—despite the fact that they may inhibit change—because bonuses are linked to SLAs associated with them.

    Metrics must be actionable.

    Useful metrics must enable DevOps leaders to make better decisions about workflows, incentives, policies, training, tools or some other “lever” of transformation. Not every such decision will be a game-changing one—but incremental gains in knowledge lead to incremental improvements in process, which collectively do change the game.

    In my next blog, I’ll discuss the metrics that DevOps leaders are using today to successfully drive the digital transformation of their businesses. In the meantime, if this is a topic that interests you, be sure to request a free copy of my book, “DevOps for Digital Leaders.”

    — Aruna Ravichandran

  • Predictable Progress: Tracking DevOps Maturity Metrics

    Predictable Progress: Tracking DevOps Maturity Metrics

    As DevOps methodologies continue to evolve, we’re naturally seeing an increased emphasis on the cultivation of related maturity metrics.

    For instance, in the 2016 State of DevOps Report, researchers with DORA took an initial stab at creating relevant ROI measurements.

    Similarly, a new report issued by researchers Coleman Parkes, sponsored by CA Technologies, offers additional evidence that as agile and DevOps processes continue to expand, today’s practitioners are increasingly employing metrics to optimize their efforts.

    The report, titled, “Accelerating Velocity and Customer Value with Agile and DevOps,” finds some form of adoption among nearly every enterprise, with 87 percent having adopted agile to some degree and 73 percent leveraging DevOps.

    Related to the overall maturity of these practices, those organizations that are moving to embrace the techniques most broadly are predictably seeing greater returns. Yet, despite the widespread adoption, only a third have deployed either practice widely, according to the research.

    Percentage of Advanced Adopters Reporting Improvements in Customer Experience

     

    Based on a survey of 1,770 senior business and IT executives, the study also closely investigates the specific KPIs that organizations are using to gauge how agile and DevOps initiatives are translating into customer interactions.

    Most notably, “advanced” agile and DevOps adopters have observed more than an 87 percent improvement in overall customer experience, compared to “basic” adopters.

    In terms of how those efforts are affecting business drivers, Coleman Parkes finds:

    • Advanced agile practitioners are experiencing a 40 percent improvement in time-to-decision, compared to 33 percent for basic users.
    • Advanced DevOps adopters have noted a 42 percent improvement in speed-to-market, compared to only 24 percent for basic practitioners.

    Key Performance-Invoking Indicators

    To that end, the research highlights specific KPIs that organizations are using to measure the impact the agile and DevOps practices are having on customers. In fact, the study finds that between two thirds and three quarters of all the organizations surveyed have established such measurements.

    At least half of these focus on business or customer-related metrics, reflecting how “critical software has become to businesses’ ability to achieve their core mission,” Coleman Parkes researchers note.

    Among the mostly widely invoked metrics, include:

    • Business growth (specifically revenue): over 73 percent
    • Operational or process efficiency: over 73 percent
    • Employee productivity: over 73 percent
    • Customer experience: over 71 percent
    • IT-related costs: over 71 percent
    • Speed-to-market: over 71 percent
    • Customer satisfaction (via NPS): over 70 percent
    • Time-to-decision: over 70 percent
    • Quality of development: over 70 percent
    • Employee retention: over 64 percent

    If increasing applications delivery speed and overall organizational velocity are, in fact, the primary goals of engaging agile and DevOps, the research bears out that the techniques are having the desired effect. After all, the survey finds that time-to-decision and speed-to-market are being accelerated among approximately a third of the organizations studied.

    Related to other notable KPIs, Coleman Parkes highlights that a full three-quarters of adopters have moved the needle on overall customer experience, along with employee recruitment and retention, a significant benefit that was also highlighted in the State of DevOps Report.

    Along with positively impacting customers, the reality is clearly that agile and DevOps are also making a big difference in allowing companies to improve the atmosphere for their workers—a huge benefit when you consider the shortage of talent in IT today, and how much it typically costs to attract new employees, just as DORA’s researchers noted.

    Other key findings by Coleman Parkes include “before and after” metrics that many organizations are observing related to their continued adoption and advancement of the involved methodologies themselves.

    I encourage you to download the entire report, and leverage it to measure how far you’ve come with Agile and DevOps transformation—and where you may need to re-examine your efforts.

    — Aruna Ravichandran

  • DevOps Ascending: Guardrails, Not Guidelines – DevOps For Digital Leaders

    DevOps Ascending: Guardrails, Not Guidelines – DevOps For Digital Leaders

    The season of DevOps is upon us, if not only related to widespread adoption, then certainly in terms of industry emphasis.

    This week, at the CA World ’16 conference, my colleagues and I will be interacting with thousands of practitioners, all of whom have evolving DevOps narratives. Meanwhile, CA was a founding sponsor of the AllDayDevOps conference, held earlier this week, attracting more than 12,000 registered attendees worldwide.

    These events come on the heels of a successful DevOps Enterprise Summit held last week in San Francisco, where many of the industry’s brightest minds gathered to discuss and debate the current state of play. If activity is a reliable indicator of relevance, the growing noise around DevOps adoption has been amplified into a deafening roar.

    All of this stands as further evidence that DevOps is no longer an emerging phenomenon, but rather a ubiquitous movement transforming the manner in which nearly every organization builds and operates software.

    As a result, the DevOps movement is finally ready to move from initial framing of emerging techniques into the domain of defining best practices. Yet, at the same time, given the fluid nature of DevOps itself and its unique correlation with the individual needs of every business and environment, even this notion of best practices is one that mandates flexibility.

    Increasingly, we hear leading experts in the field espouse the need for these practices, but those that serve as “guardrails” that set useful parameters for DevOps adoption and refinement rather than concrete guidelines that dictate specific requirements.

    Such is the tone and goal of the new book that my co-authors Kieran Taylor, Peter Waterhouse and I released this week to help practitioners understand common DevOps challenges, methods and milestones.

    Titled “DevOps for Digital Leaders,” the book examines the genesis and increasing maturity of DevOps, all with the explicit intention of helping to inform further refinement among today’s organizations. After reading it, I’d argue that any practitioner, from total neophytes to hardened DevOps vets, should walk away with a greater understanding of how to bring today’s best practices to bear within their own work.

    Beyond looking at precisely how DevOps has grown in response and parallel to use of agile methodologies in the world of manufacturing thereby transforming the modern software factory, the book offers specific examples of leading adopters that are using these techniques, and related tooling, to re-engineer the very DNA of their business.

    From the core competencies of building, testing and deploying applications to continuous monitoring and management of software to help stay attuned to changing customer expectations, we sought to offer a practical guide to DevOps transformation. Other chapters touch on finer points related to key topics including creation of success metrics and measurement of ROI.

    In one of the analogies invoked throughout the book, we highlight how DevOps-empowered software factories are not unlike the engineering, design and race day teams present in the world of today’s Formula One racing series.

    Pushing themselves and constantly innovating on the fly—clawing for every second in the name of winning the overall title—the comparison fits nicely as we watch this season’s contenders fight it out down to the final race in Abu Dhabi later this month. As with winning customers in today’s applications economy, clearly every little bit counts.

    Again, no matter what—or how much—you think you know about DevOps, we’d like to think that there are tactical “learnings” sprinkled throughout the book that will broaden and deepen your understanding.

    Download your copy today, and let us help drive you down the road to victory. Alternatively, you can order a hard copy of the book via Amazon, with all proceeds being donated by the authors to fund Foundation for Excellence, which provides STEM scholarships to children in India.

    — Aruna Ravichandran

  • Keep Applications Running by Testing Early, Continuously

    Keep Applications Running by Testing Early, Continuously

    Almost every product we buy or use today goes through some form of extensive testing before it’s released on the market; typically, these tests are designed to replicate the worst-case scenarios that a product may encounter under real-world conditions.

    Automobiles, for example, go through extensive hot and cold weather testing in remote locations, and are driven over obstacle-laden tracks designed to push the limits of their durability. Even seemingly mundane objects such as children’s toys are exposed to abusive simulations to ensure that they don’t harbor any potentially hidden and harmful effects.

    On the current landscape, applications are the face of your business, often serving as the battleground where consumer loyalty is won or lost. Given this reality, it seems crazy to think that any applications are released having only been exposed to a minimum of tests. Yet, in the name of getting to market quickly, that’s far too often the case.

    As disciplines including agile and continuous delivery continue to increase the speed at which development teams can release software, it’s easy to put the crunch on testing in the rush to get an update released. But that’s a costly mistake in the long run, as research has shown that it is 50 percent less expensive to find an error in testing than to address the same error once it has reached production, and more than 90 percent cheaper  when addressed in the development/coding phase.

    Given the dramatic cost savings of addressing errors in development, clearly the goal of any organization focused on continuous delivery should be to shift testing left so a greater number of defects are found early in the development process. This is what we refer to as continuous testing, and it’s a huge part of the reason why I am so excited about CA’s acquisition of BlazeMeter.

    Performance load testing should be a core element of any continuous testing strategy because it is a critical factor in determining system reliability. Similar to the worst-case scenario testing that consumer products undergo, performance testing can quickly uncover applications defects that might otherwise go unnoticed. After all, the manner in which a system operates with a few hundred concurrent users may be very different than how it runs with thousands of users.

    BlazeMeter enables today’s development, operations and DevOps teams to conduct this critical performance testing earlier in the development process, saving countless hours and resources. The company is also competing—and winning—in a highly competitive marketplace where it has been pitted against much larger, more entrenched competitors.

    Part of the reason BlazeMeter is winning is its focus on offering a SaaS-based solution that is highly cost-effective and easy to engage. Using this model, customers can try  out the testing software with a few clicks, and then quickly move forward to broadly adopt the system. Another catalyst to its success: BlazeMeter was built using open source tools, such as Apache JMeter, used widely in the development community for performance and load testing. As a result, BlazeMeter can easily run and analyze JMeter and other open source-based load tests—such as Selenium, Gatling and Locust—within a commercial-grade performance testing system.

    This combination of a SaaS delivery model and incorporation of open-source tools, augmenting strong performance testing functionality, make BlazeMeter very complementary to CA’s existing continuous delivery solutions, including its CA Service Virtualization, CA Test Data Manager, CA Agile Requirements Designer and CA Release Automation products.

    At CA, we’re committed to providing the most comprehensive, integrated and open continuous delivery ecosystem, allowing our customers to release swiftly, test agilely and constantly increase reliably. The addition of BlazeMeter will further enable customers test continuously, thereby increasing agility and the overall quality of their software, to ensure that applications don’t break down on the side of the virtual road.

    — Aruna Ravichandran

  • State of DevOps Report: Reaping Results

    State of DevOps Report: Reaping Results

    The newly released 2016 State of DevOps Report concludes that DevOps is “no longer a mere fad or buzzword,” but rather an established set of practices and cultural patterns. Yet, a closer look at this year’s findings reveals progress is far more substantial than any matters of perception.

    The report was conducted and analyzed by the experts of DevOps Research and Assessment (DORA) and sponsored by CA Technologies. Among its findings is proof that high-performing organizations that have embraced agile and DevOps methodologies deeply are outperforming massively those that have not. And the gap is widening.

    The State of DevOps Report also shows leading DevOps practitioners are appreciating benefits that span nearly every area of their business. If I know my colleague Gene Kim, he will assert this point exuberantly on our upcoming State of DevOps Report Webcast.

    It’s noteworthy that high performers are surpassing incredible milestones related to core elements of the applications life cycle. Examples include deploying code 200 times more frequently than low performers, driving 2,555 times faster lead times, 24 times faster recovery times and producing three times lower change failure rates. This is evidence that DevOps is having the desired outcome on applications delivery, software quality and organizational velocity. It also shows organizations truly committed to wholesale transformation of culture and process are actively succeeding.

    At the same time, the State of DevOps Report shows employees working at organizations enjoying this success are more satisfied with their jobs and more closely engaged with their companies. Chalk this up to the halo effect of operating in a fundamentally more sensible, innovative fashion. Cultural change is undoubtedly the greatest hurdle to DevOps advancement. Here’s proof that if you can navigate the process, your workers will be happier and more productive.

    Using Employee Net Promoter Score (eNPS) as a barometer, the report finds that employees in high-performing organizations were 2.2 times more likely to recommend their organization as a great place to work. They also were 1.8 times more likely to recommend their team as a great working environment. Previous studies have found that such an air of positivity can be directly correlated with improved business outcomes, researchers noted.

    Shifting the manner in which you build applications to better adapt to changing customer expectations is a necessity. That’s what began driving this movement years ago. That said, proof it also makes it easier to keep valued software engineering and IT operations employees should not be overlooked.

    In addition to positive conclusions on the impact of continuous delivery and improvement of test data management, the report offers some important metrics related to return on investment. This is nothing short of landmark, as for decades we have struggled to make a hard case for IT investment beyond implied competitive advantage. It actually was impressive that last year’s report highlighted such a “predictive relationship between IT performance and organizational performance.”

    One such new measurement of note is that high-performing teams occupy the least amount of time responding to unplanned work and rework—efforts widely recognized to be results of flawed processes. On a related note, DevOps leaders spend 49 percent of their time on new work‚ surely, in many cases, the next “must-have” feature which impacts competitive standing and, ultimately, the bottom line.

    This report is fantastic. It reaffirms powerfully that so many of the promises made about DevOps over the last few years are neither hype, nor overstated. The gap between the leaders and laggards is growing. Industry landscapes and business models are being redefined.

    This is critical transformational work that every organization must adopt to survive.

    Are you on board? Download the full State of DevOps Report here.

  • Piecing Together the Elements of DevOps Transformation

    Piecing Together the Elements of DevOps Transformation

    Sometimes when a concept as transformative as DevOps comes along, it’s easy to forget that subsequent adoption represents a highly complex and iterative process.

    While it’s clear that some organizations are already deeply leveraging DevOps to embrace the Applications Economy and Digital Transformation, it’s no surprise that for many organizations this process has only just begun.

    So where have today’s organizations made the most progress, and what are the most significant challenges that practitioners continue to face?

    To get a better fix on the current state of DevOps maturity and isolate the specific areas where most organizations must continue to focus their attention, researchers at Freeform Dynamics have created a matrix of practices key to measuring progress, dubbed the “DevOps Jigsaw”.

    Drawn from the same survey data recently surfaced to profile leading-edge “Digital Disruptors” that have already wrapped their arms around DevOps advancements, this latest report, sponsored by CA Technologies, offers a crucial view into the current environment.

    It’s worth noting that of the 1000-plus respondents included in this particular data slice, the vast majority, or 80%, are categorized as “Early or Limited” adopters, with only 20% recognized as “Advanced”.

    Of the nine individual DevOps “puzzle pieces” examined in the report, experts grouped these efforts under three umbrella categories: “Key Enablers and Controls”, “Skilled and Collaborative IT”, and “A Business Led Approach”.

    Business Led Approach – This subset encompasses strategy and objectives required to tie DevOps adoption to larger business goals, including:

    • Well-defined strategy and objectives – In this key planning stage of DevOps transformation, 55% of respondents said that much of their work is “already done”, while another 36% charted it as “a work in progress”.
    • Business stakeholder education – Roughly 33% of respondents said that this informative step in DevOps adoption is complete, with a larger majority of 43% noting that they are still working to advance this process.
    • IT-business alignment of priorities – In getting stakeholders on the same page around business and DevOps direction, some 37% indicated that they are nearly done, with 43% of still attempting to embrace these efforts.

    Skilled and Collaborative IT – This genre of requirements covers the cultural and automation capabilities necessary to foster DevOps success, including:

    • Relevant IT knowledge and skills – Related to the development of DevOps expertise, 39% of respondents feel that they’ve already made the grade, with 40% conceding that they have more homework to complete.
    • Cross-functional IT processes – As much as any related initiative, DevOps mandates tighter internal collaboration, with only 35% of respondents having achieved this goal, and 45% citing the need to keep pushing.
    • Cultural harmony within IT – On a related note, DevOps teams must work together smoothly to succeed, with a mere 29% of respondents claiming such parity, offset by 46% that acknowledge a teamwork deficit. 

    Key Enablers and Controls – This category of DevOps maturity addresses key issues of technology, solutions acquisition and IT security, including:

    • Right infrastructure and tooling – Efficient tooling and infrastructure are another key DevOps enabler, with some 37% of respondents feeling good about their current solutions, and 42% still looking for answers.
    • Right suppliers and support – In sourcing the products and services required to advance DevOps process, only 33% of respondents said that they’ve found the right providers, with 46% still seeking additional help.
    • Security and compliance measures – No area of IT can be addressed without consideration of security and audit issues, with 35% of respondents comfortable with their controls, and 46% in need of added defenses.

    Given these overwhelming results, it’s clear that most organizations still have a long way to go in achieving greater DevOps maturity; at the same time, there’s also plenty of evidence that today’s practitioners have already made a good deal of progress.

    Comparing your efforts to these established “DevOps jigsaw” puzzle metrics should help inform future strategy in creating a more cohesive framework of people, process and technologies, and deliver DevOps success.

  • Microservices, Hyper-Complexity, and Automation

    Microservices, Hyper-Complexity, and Automation

    Imagine building a human being one DNA codon at a time. Sure, you might be able to build a pretty awesome individual. But it would take a long, long time. And you’d only be able to build one. Worse yet, if you didn’t get your person exactly right on your first try, it would take you another long, long time to come up with Version Two.

    Unfortunately, this is exactly the position many companies are putting themselves in with microservices.

    Yes, microservices theoretically enable companies to adaptively recombine their digital DNA to build the increasingly awesome apps. But without better automation, they won’t be able to do so very quickly—or very reliably.

    Hyper-Complex DevOps

    Microservices introduce significant additional complexities into the DevOps lifecycle. Each microservice has its own data mappings, APIs, latency/performance dependencies, etc. So, while microservices may offer genuine advantages in terms of adaptability and re-use, they also multiply the total number of discrete DevOps tasks—especially vis-à-vis test/QA—by at least one full order of magnitude.

    This is a serious problem for shops that aren’t sufficiently well-automated, because any task—or, as is much more often the case, any handoff between tasks—that is not well-automated will quickly turn into a bottleneck.

    The resulting adverse consequences of insufficiently automated microservices DevOps can thus include slow delivery, excessive cost, and mistakes. Worse yet, chronic bottlenecking will likely create a pervasive organizational disincentive that fundamentally undermines the microservices initiative itself.

    What Automation Entails

    Thorough, effective DevOps automation is, of course, more easily aspired to than achieved. To implement the kind of end-to-end DevOps automation required for microservices success, organizations have to get three things right:

    • Process mapping. Effective automation requires an exhaustive definition of process. This exercise is not for the faint of heart—because most organizations have many pockets of ad hoc process at various points in the DevOps cycle. Test/QA tends to be especially variable based on factors such as the delivery pressure teams are under at any given time.
    • Automation technology. Development teams often use scripting and other technologies to automate various DevOps tasks. An overly fragmented automation environment, however, can be difficult to orchestrate and manage. So the move to microservices may necessitate a more holistic integrated automation environment.
    • Leadership and culture. A well-automated DevOps process alone is insufficient for microservices success, because it is unfortunately not true that “if you build it, they will come.” On the contrary, a strong leadership commitment is essential for changing the way people work.

    The application economy is putting development organizations under intense pressure to be more productive and responsive to ever-changing market dynamics. Microservices offer one possible way to respond to that pressure. But microservices won’t work without a much more rigorous approach to DevOps automation. IT leaders may therefore want to focus on automation first and microservices second, instead of the other way around.

    Unless, that is, you only want to build one app using microservices—and then maybe build a slightly better one a few months later.