Progress in achieving the maximum benefits to be gained from investing DevOps processes appears to be held in check by a failure to master the art of continuous testing. A new survey of 923 IT and testing professionals published this week by CA Technologies suggests that only 20 percent of the respondents could be classified as “leaders” when it comes to implementing continuous testing.
A full 93 percent of respondents identified testing automation as important, but only 1 in 5 respondents said they achieved a high level (80 percent or more) of test automation coverage. Seventy-five percent said continuous testing is important as well. Unfortunately, most of the respondents continue to rely on manual processes. In fact, respondents attributed continuous testing challenges to the lack of automation in almost every aspect of the testing process, including generation of test cases through test execution to critical activities such as fast, safe and efficient test data management.
Scott Edwards, senior director of product marketing for CA Technologies, says failure to automate continuous testing will only lead to an increased DevOps divide. Organizations that have automated DevOps processes not only release more code faster, the report find they enjoy advantages such as:
• 2.3 times more likely to have succeeded in left-shifting testing activity to developers;
• 2.6 times more likely to reduce defects by more than 50 percent;
• 2.4 times more confident in quality of output;
• 1.9 times more confident in speed of delivery; and
• 3.9 times more likely to be working in an organization exhibiting rapid revenue growth.
Edwards says those results indicate that organizations that are further down the DevOps process will enjoy a substantial competitive advantage at a time when organizations of all sizes are realizing how dependent they are on software to differentiate themselves. Organizations may be operating in a variety of vertical industries, but increasingly they are becoming software companies that happen to specialize in a specific area, says Edwards.
Making that shift requires IT organizations to move away from thinking about developing applications as an art to building what amounts to a software factory, he says. The factory approach not only eliminates silos in the DevOps process, it also should ideally allow developers to continuously test their own applications. Of course, the biggest issue with testing may very well be the simple fact that most developers build applications today using notebooks and workstations that don’t accurately reflect the complexity of the data center environments those applications will ultimately be deployed in. However, thanks to the rise of technologies such as service virtualization and cloud-based development environments, it has become easier to more closely mimic production environments.
The degree to which an absence of continuous testing impedes both the rate at which applications are deployed in a production environment will naturally vary by organization. But the one thing that is almost certain is that organizations that have mastered continuous testing are much more likely to deliver a quality application experience than those that don’t.
This is testing unleashed, going hand in hand with agile development to accelerate quality into production.
Wayne Ariola, CMO of 
Sandeep Johri, CEO of Tricentis, has more than 25 years of experience as a technology executive and serial entrepreneur—including roles at Hewlett-Packard, Appcelerator, Oblix (acquired by Oracle), Blue Lane Technologies (acquired by VMware), and numerous other startups in Silicon Valley and India. Johri earned an MBA from Stanford University, a Master’s degree in Industrial Engineering from Wayne State University, Detroit, and a Bachelor’s degree in Mechanical Engineering from Pune University, India. Connect with him on 
