devopsy.kinsta.cloud

Category: Continuous Testing

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

  • 4 Ways to Tackle Common Test Automation Pitfalls

    4 Ways to Tackle Common Test Automation Pitfalls

    The demand for higher-quality products and new features delivered at breakneck speeds has the average engineer struggling just to stay afloat. With demands rising but resources not always following suit, engineering teams are embracing automation as a key testing option.

    According to a report by advisory firm KPMG, test automation increased by 85 percent from 2015 to 2017 across all industries. Automation is one way to address this accelerated development schedule, but it can also introduce other problems. To ensure that doesn’t happen, there are four common pitfalls that engineering teams need to look out for.

    Problem No. 1: Right Job, Wrong Tools

    It’s tough to build a shed with a spatula and a sewing needle. Likewise, you can’t build a successful test automation suite if the tools used don’t fit your company’s testing strategy.

    The solution: Pick the right framework for your suite.

    The first question you need to ask yourself is: Should you build or buy your test automation framework? The answer likely depends on how mature your testing structure is. Smaller teams new to the technology typically are best-served starting with an off-the-shelf enterprise solution. This approach makes it easier to get up and running quickly, while having the support to handle software bugs along the way.

    Larger, more mature testing teams may have more to gain from an open source automation framework. This cost-effective approach provides flexibility and customization, but often requires advanced technical expertise and the luxury of time to build such a solution.

    Problem No. 2: Living Fast, Dying Young

    Many automation projects never make it off the ground because teams move too fast and automate too much, too soon. Inserting automation into the delivery pipeline prematurely will cause workflow and build issues more often than not.

    The solution: Build test automation into your pipeline incrementally.

    It may be tempting to run full speed as soon as you are set up, but the truth is you need to take your time for developers to build confidence in the testing process. Once that confidence is established and you’re receiving valuable feedback, you will have the full team’s support as you continue to scale your automation strategy.

    Problem No. 3: Automation Without Documentation

    Automation can make testing easy. Sometimes too easy, to the point where engineers will simply kick back a failed test without looking into the reason it failed in the first place. Oftentimes, these failures are the result of an outdated testing script or an issue with the test environment, and not with the code at all.

    The solution: Triage automation test results just like manual bugs.

    Instituting an intermediary triage stage will save your engineering team time in the long run. There’s no need to treat your processes for automation results any different than you would bugs found manually. Having access to materials such as activity logs and screenshots will only help to strengthen and speed the triage process.

    Problem No. 4: Setting and Forgetting

    Automation makes testing quicker and easier. That’s why you invested in it in the first place, right? Unfortunately, there is more to such solutions than just plugging them in and letting them run.

    The solution: Take maintenance into account when building your framework.

    Many teams are surprised by the maintenance required to keep test automation suites running smoothly and effectively. They shouldn’t be. Solutions need to be updated continually to address user interface changes, new operating systems and browsers, and automation library changes. The maintenance of third-party dependencies also needs to be considered before automation is used for production test runs.

    Maintaining your automation suite is a continuous process. Make sure you have enough testing resources—either in-house, outsourced, crowdtesting or a combination of all three—to ensure your efforts don’t go to waste.

    Measure Twice, Implement Once

    Make no mistake, automation is more than a quick fix. If you are going to spend the time and money necessary to build a successful strategy, you should make sure to set yourself up with a solid foundation. A framework that is thrown together within days but falls apart in hours will only serve to make automation seem like an unpalatable solution within your testing department. Having the right partners in place and the right solution from the start will help automation become a reality in your company, and can bring the promises of speed and flexibility to life.

    — Drew Horn

  • Continuous Testing at More than 200 mph

    Continuous Testing at More than 200 mph

    “The old principle of tinkering with an instinctively designed car has long since been superseded by systematic testing of every major component and structure – both before and after the car is fully built and ready to race.”

    —“Understanding F1 Racing,” Formula1.com

    While speed is the most obvious shared goal between racing and software development teams, you might be surprised at the striking similarities between how these teams achieve that goal. Each relies on top talent, funding and innovation to reach the speeds at which their competition cannot, but there’s another oft-ignored component that’s quickly becoming one of the greatest differentiators between first place and everyone else: testing.

    The way F1 teams test their cars has gone through a transformation just as monumental as what we’ve seen in the software world over the last decade. The ability to continually test every major component and structure of a race car or piece of software is crucial, but the ability to quickly respond to the data that comes out of those tests is just as important. It’s continuous testing before, during and after the rubber meets the road—or the software hits the marketplace or app store—that determines the winners.

    The Evolution of Testing

    In the aforementioned Formula1.com article, an all-too-familiar scenario to those in the software world is described:

    “By midway through the first decade of the 21st century, a typical Formula 1 testing programme had become a major exercise in both manpower and logistics…spiralling costs, which invariably resulted in development work that was invisible and meaningless to fans …”

    Replace “Formula 1” with “software” and the above excerpt is just as factual for those who don’t know the struggles of yesterday’s approaches to software quality, and just as painful for those who personally experienced them. Manual processes, insufficient test automation technologies, shared unrealistic environments, lack of trust between teams and out-of-control costs are constraints of the past for today’s top-performing teams, but they remain for many still relying on legacy testing approaches and technologies.

    What Continuous Testing Looks Like Today

    Some of the racing enthusiasts from Tricentis’ leadership team were invited earlier this year to meet the Haas F1 racing team during their 2018 preseason testing at Circuit de Barcelona-Catalunya. We witnessed firsthand the remarkable transformation that Haas and F1 racing as a whole have embraced. Track testing is where, as Formula1.com says, “the steady evolution that happens to all Formula One cars during the course of their life begins, a constant improvement of tiny details and setup.” This constant evolution and iterative, rapid innovation is exactly the goal of high-performing software organizations today.

    After spending the day at the track, we noted the following approaches and methodologies that we see in innovative continuous testing and DevOps practices today:

    • F1 cars are continuously delivering data and feedback to the organization.
    • The cars have millions of data points and thousands of sensors, providing actionable data.
    • These data points and sensors are grouped into core processes, and isolated by key questions.
    • Test runs are done with the cars in as realistic environments as possible.
    • The speed at which that data can be received, interpreted—and most importantly—acted upon, is the difference between high-performing and low-performing teams.
    • The tiniest changes to the car make enormous differences. Nobody is showing up in an entirely different car, they’re making minute changes.
    • These cars cannot fail on race day or the race is lost.

    Today’s high-performing organizations are treating the delivery of software in the exact same manner as the world’s leading racing teams. Innovative technologies are being put into each stage of the delivery life cycle that provide invaluable feedback that helps enterprises better understand risks to their business. Being able to rapidly analyze—and act on—this amount of data has become imperative for avoiding disruption. Testing software in environments that are as realistic and as close to production as possible is mandatory, and those environments need to be available when they’re needed most. These needs have driven the adoption of what’s known as continuous testing—an evolution of traditional test automation that has proven unable to support software’s increased complexity, and expectations around the pace of development and delivery.

    What Questions Are F1 Racing and Software Enterprises Asking?

    While each have their own unique challenges, there are many similar questions that need answers—in real-time, and at any development, testing, or release stage—that the racing and software industries share.

    • How fast are we today and how fast will we be tomorrow?
    • How fast is our competition, and how fast will they become?
    • What are we measuring today that’s impacting the business?
    • How quickly can we introduce change?
    • What data are we using to validate that change?
    • What’s the risk of not changing?
    • How low can we get our mean-time-to-repair?

    The fastest time to market—or time to the finish line—is how you win the race. Your developers or engineers don’t just build the fastest car, or the most popular app to today’s standards and then call it a day. They’re constantly looking at what speed, UX, functionality or accessibility will be needed to be the best tomorrow, and then iteratively adding those improvements as quickly as possible. They don’t stop engineering, ever. And testing shouldn’t be performed any differently. Each new feature, condition, region, device, persona, currency, language and OS that your app needs to offer, integrate with, accept or run on is going to need continuous testing before, during and after that app’s release.

    Whether you’re responsible for the quality and performance of a billion-dollar race car or the applications for a billion-dollar enterprise, speed and quality are your path to the winner’s circle. Continuous testing ensures that you remain there.

    — Noel Wurst

  • DevOps Leaders vs. Laggards: Continuous Testing is a Key Differentiator

    DevOps Leaders vs. Laggards: Continuous Testing is a Key Differentiator

    New Forrester research, “The Definitive Software Quality Metrics for Agile + DevOps,” reveals what software quality metrics and practices differentiate DevOps/Agile leaders from DevOps/Agile laggards. The study was commissioned by Tricentis and led by Forrester VP and Principal Analyst Diego Lo Giudice.

    The study found that companies with the most successful Agile and DevOps initiatives do a number of things differently than their peers:

    • They are transforming software testing into continuous testing by adopting five core practices, such as automating end-to-end functional testing and integrating testers into cross-functional teams.
    • They are almost twice as likely to consider automating the software quality process to be a “critical business differentiator.”
    • They are significantly more likely to have high levels of automation for key testing and QA processes (test case design, functional test automation, test data management, etc.).

    However, only 26 percent of firms with active DevOps adoptions are currently practicing the five core continuous testing practices shown to separate leaders from laggards. And, as the infographic below highlights, many CXOs have a severely inflated sense of their firms’ DevOps and continuous testing maturity.

    Different DevOps Quality Metrics

    The study also identified the 20 most important Agile and DevOps quality metrics that separate DevOps/Agile experts from their less advanced peers. Forrester examined 94 quality metrics that were classified into four categories. Key findings include:

    • Understanding of business risk is the most important determining factor of DevOps and Agile maturity.
    • Experts focus primarily on contextual metrics (e.g., requirements coverage) while others focus on “counting” metrics (e.g., number of tests).
    • Experts are more likely to measure the user experience across an end-to-end transaction while others rely on application-specific or team-specific metrics.

    To determine the “most important” metrics, the research surveyed 603 global enterprise participants responsible for their firms’ Agile and/or DevOps strategies. Participants were first asked a variety of questions to assess the maturity of their Agile/DevOps practices. Next, they were asked what quality metrics they actually used, then asked to rank the value of each metric they regularly measured. Results from the Agile/DevOps experts were later separated from, and compared to, those of the “other” respondents. The most important metrics were determined by analyzing the experts’ value rankings for the metrics that they actually used.

    The Risk ‘Blind Spot’

    Risk was a prevalent thread throughout the various parts of the report. Although risk-related metrics did not rank high in overall popularity, Agile and DevOps experts measure them significantly more frequently than the non-experts did. In fact, Agile and DevOps experts consistently ranked risk-related metrics among their top three most valuable metrics in different phases and categories.

    Interestingly, most firms (80 percent) believe they deliver within acceptable business risk, but fewer than a quarter state that their QA and testing processes completely cover business risk. Only 15 percent of respondents say that their test suites reliably provide a good indication of business risk.

    From Tricentis’ analysis of Global 2000 companies’ test case portfolios, we’ve found that most test suites actually cover about 40 percent of the organization’s business risks. However, these test suites have a 67 percent level of redundancy on average—meaning that over two-thirds of the tests don’t actually increase their business risk coverage. This might be one cause of the disconnect: people assume that more tests means more business risk coverage, but that’s often not the case.

    Another likely cause is that most test results focus on pass/fail status, which doesn’t provide the required insight into whether the release has an acceptable level of risk. As companies move towards continuous delivery and automated release processes, real-time insight into business risk becomes critical for promoting acceptable releases as rapidly as possible while stopping potentially-damaging ones from proceeding down the release pipeline.

    Details and Additional Findings

    If you want to learn more about how Forrester arrived at these findings—and see how the 94 quality metrics ranked—you can read the complete “Definitive Software Quality Metrics for Agile+DevOps” on the Tricentis website.

     

    — Wayne Ariola

  • Testing Heroes to Recognize Individual Contributions to Continuous Quality

    Testing Heroes to Recognize Individual Contributions to Continuous Quality

    Testers have always been great at championing the superstars within the testing community, but who else outside of testing has been listening? Tricentis has launched a campaign aimed at making sure the rest of the world knows just how invaluable testers are, especially in today’s hyper-accelerated release cycles, and with user experience and customer satisfaction becoming the enterprise’s success metrics of choice.

    Testing Heroes will be an annual program in which software testers can be nominated for a variety of award categories, two of which—Tester of the Year and Test Manager of the Year—will be awarded a free trip to Vienna, Austria, to accept their award onstage at Tricentis’ Accelerate 2018 conference.

    Testing Heroes is split into six different categories:

    • Tester of the Year
    • Test Manager of the Year
    • The Explorer
    • The Automator
    • The Performer
    • The Risk-o-Meter

    A Unique Twist

    While most winners are decided on by the awarding organization, the 2018 Testing Heroes will be determined by the level of support from each nominee’s surrounding organization and network. The final round of Testing Heroes will be a voting round. Those whose nomination makes it to this round will be provided with a variety of ideas for encouraging others to rally behind their candidate to get them the recognition they deserve. There are myriad ways to spread the word in your office about the outstanding work being done by one of your own:

    • Encouraging others in your company to vote through Slack, email or an internal newsletter.
    • Recording a short, campaign-style video to share through social media.
    • An iPad loaded to the candidate’s voting page placed in a break room, kitchen or other communal area.
    • Desk-to-desk canvassing (to those who don’t mind you popping in unannounced!).

    One of our initial goals for this campaign was to learn more about the amazing contributions by those in the software testing community, but we quickly realized there is a bigger reward here: When a nominee’s peers bring to light the amazing work being done by one of their own during the campaign process, software testers’ efforts hopefully will be more understood and appreciated by a larger audience than they are currently. This is the real goal of Testing Heroes.

    We encourage you to nominate and then rally behind your own Testing Heroes today!

    — Anand Akela

  • How Automated Testing Can Help the Latest C-Level Issue: Quality Software

    How Automated Testing Can Help the Latest C-Level Issue: Quality Software

    Every year developers gather in San Jose for Apple’s annual WWDC, where software quality is typically a key point of discussion. This year, the company progressed from its 2017 buggy software debacle in full force—it’s a heads-on effort, marked by the industry leader naming 2018 its year of software quality.

    This raises some larger questions around the software industry itself. Has modern IT become so complex that the concept of “bug-free” software is no longer realistic, particularly for a company the size and scale of Apple? Or did Apple recognize this pain point and shift its focus to  software quality in an effort to be the harbinger of change?

    Of course Apple isn’t alone when it comes to the struggle with quality. Buggy software from organizations across the board persists today, but it simply cannot be ignored. Poor quality software creates long-term effects of waning customer confidence, brand erosion and revenue loss. These are very real concerns for the entire C-suite. Quality is no longer an issue limited to Dev or Ops—it’s critical to every organization’s success.

    For example, consider the fact that 88 percent of consumers say they’d abandon an app if they encountered bugs or glitches (source: QualiTest). Nearly half those people are likely to discontinue using an app entirely if they experienced a bug every day. And 32 percent say they would abandon an app the moment they encountered a glitch.

    The cost of acquiring a new customer is 5 to 25 times higher than retaining an existing one. Getting software right the first time through a culture of continuous testing is critical to every business, as it translates directly to retaining customers.


    Budgets for QA and testing are expected to rise to 40 percent in 2019 from 31 percent this year. However, one-third of developers still admit they don’t have enough time to properly test before software deployments. As everyone in this business knows, developers run fast. It is of the utmost importance to ensure software actually works well and will provide the expected customer experience. The garbage in, garbage out model is not an option. To fill the gap, more and more organizations are turning to automated testing.

     

    Any repetitive, labor-intensive task is typically a good candidate for automation, including those associated with testing. Automated testing involves execution of prescripted tests on a web or mobile app, with test data accessible in detailed reports to help development teams quickly identify issues in the code, as well as any issues in the automated scripts themselves.

    Performance of the automation framework and its scripts is crucial, and should be a high area of focus. Prescripted tests fall into different categories that comprise the test pyramid. They include:

    1. Unit Testing
    2. Code Coverage
    3. Code Syntax Checks
    4. Security Tests and Checks in Code
    5. Integration Tests
    6. UI Tests for Web, Hybrid and Mobile Apps – on multiple cross browsers on various platforms and OS versions

    Once created, automated tests can be run again and again at no additional cost—which is particularly helpful when source code is changed. Since it is much faster compared to manual testing, automated testing translates into considerable time and cost savings.

    Automated testing is especially valuable given the breadth of browsers, operating systems and devices in the marketplace. Consumers today run apps on a countless number of these, so developers must have access to every combination imaginable to deliver the highest level of performance. What good is your app if it doesn’t run (and, by extension, monetize) properly when it gains popularity in some part of the world with users on a different platform from the one you tested on?

    Ensuring that software works on as many options as possible allows a business to reach a maximum number of users. This is where automated testing can help, by executing thousands of different complex test cases during every test run. Testing efficiency can be achieved by executing tests in parallel or grid, and executing CI/CD jobs in parallel. For example, one code is deployed in staging, teams can simultaneously start jobs to test against browsers and cross-browsers, apps and localizations, etc. The result is attaining coverage that would be impossible through manual tests. Teams can scale apps fast, maximize CI/CD workflows, and ship higher quality software faster.

    Attaining CI/CD and automating all the tasks involved in creating a real-time test system is one of the arenas in which the next big software battle will be fought. More importantly, as software continues to eat the world, the C-Suite can no longer ignore the impact of poor quality software devouring your business. Apple’s bold step of identifying 2018 as the year of software quality is a call for other organizations to do the same.

    Automated testing is a key piece of the solution. How is your organization meeting the software quality challenge, and by extension, the evolving standards of customer experience?

    — Ritesh Arora

  • Selecting the Right Service Virtualization Tool

    Selecting the Right Service Virtualization Tool

    Once you have established that implementing service virtualization (SV) as a means to remove the bottlenecks formed by critical yet hard-to-access dependencies in your test environments is the way to go forward, one important decision to make is which SV tools to use for the process. Most major testing solution vendors (including CA, Micro Focus, Parasoft, Tricentis and more) offer an SV tool as part of their product range. Next to that, smaller companies have emerged for which an SV tool is their primary offering (TrafficParrot, for example). Finally, similar to the test automation ecosystem, many open source offerings have seen the light of day, most notably including WireMock and Hoverfly.

    Similar to what has been going on in the test automation world for a while now, open source SV tools seem to have become the default choice for many development teams when looking for a solution that satisfies their SV needs. In this blog post, I’d like to take a closer look at the pros and cons of this bias toward open source products, as well as provide some tips on how to decide what type of tool works best for your organization and situation.

    Benefits of Open Source and Commercial Solutions

    There are several benefits associated with the use of open source service virtualization tools in general, and these benefits apply equally to SV tools:

    • There’s no risk of vendor lock-in when you’re using an open source tool.
    • There are no license costs associated with open source tools, whereas commercial off-the shelf (COTS) tools obviously do.
    • Open source tools can be extended with specific features in case you need them (and you’ve got the development skills to do so). With COTS tools, you can file a feature request with a vendor, but that’s no guarantee when your request will be addressed (and if it will be addressed at all).

    COTS solutions, on the other hand, have benefits of their own:

    • They come with a richer set of features out of the box, such as support for a wide variety of transport protocols and message formats, etc., whereas open source tools often focus on a specific protocol and/or message format.
    • They generally have a lower level of entry, for example because they offer graphical user interfaces that make them accessible for a larger audience, whereas open source tools may require you to write code or other forms of technical specifications to get them running.
    • Professional support, both online and in house, may be included in the license purchase price, whereas open source tools offer support on a best-effort basis only, most of the time.

    This is by no means a complete list of the pros of both types of tools. Also, some of the benefits mentioned here come with a caveat of their own. For example, the lower initial cost of open source tools (no need to buy a license) is often being made undone by the additional effort it might require to build simulations that are as powerful as those that can be created by commercially licensed tools out of the box. Always consider the total cost of ownership when you’re looking to implement an SV tool, and don’t forget that this TCO extends beyond licensing fees.

    One other important aspect to understand when comparing open source and commercial tools is that they’re typically sold to different audiences and therefore, their creators must appeal to these different audiences accordingly. Open source tools are generally adopted in a bottom-up manner: they’re downloaded and experimented with by a developer or development team and, when successful, might slowly be adopted by a larger audience within the organization. Commercial tools, on the other hand, are often sold to development managers or C-level executives instead, since they hold the budget that is needed to make the purchase of licenses possible. Note that this is a generalization, and it might not work this way in your organization, but in general, I feel the above holds true. (Footnote: I have to say thanks to Tom Akehurst, the creator of the open source SV tool WireMock, for pointing out this important distinction to me.)

    So, now that we’ve seen some of the benefits of both open source and COTS SV tools, I’d like to give you some advice on how to choose the right tool for your organization and situation.

    First of all, understand that no tool is inherently better than another one (even though some tool creators and vendors seem to think otherwise, judging by their promotional and social media outings). Tool A, however, might under certain circumstances be a better fit for a given situation compared to tools B, C and D. What tool works best for a given situation depends on many factors including, but definitely not limited to:

    • The extent to which the required transport protocols and message formats are supported out of the box (better support makes for easier implementation).
    • The level of experience with SV of the people that are going to wield the SV tool.
    • The complexity of the behavior that is to be simulated.

    When you’re looking for the best fit for your SV needs, it is wise to take into account a range of tools, both open source and commercially licensed. Don’t discard the latter category because of the seemingly high initial investment due to licensing costs. As I’ve pointed out before, it might just be that the ease of implementation and the richer set of features provided by a commercially licensed tool give you a return on investment that warrants the initial expense many times over. I’ve seen this happen in practice multiple times, and that’s the reason I now prefer to include a range of tools in my test automation and service virtualization training courses, rather than focusing on a specific tool. I sincerely believe there is no one-stop solution for every situation and I’d rather show people what’s available on the market and help them see what would be the best fit for their own organization than train them in a single solution (with the risk of giving them a hammer and making them see everything as a nail).

    Also, don’t simply go with the first tool promoted by a Google search or a friendly sales representative, as there’s a good chance that it might not be the most efficient choice for your specific situation. Take the time to select a tool carefully. Conduct a proof of concept (or more than one, ideally) that demonstrates the actual value that a tool can bring you, and that proves that a tool fits your development and testing requirements and is a good match for the skill set of your development team (or anybody that is to be made responsible for creating and maintaining the simulations). Consider hiring an independent expert that can help you make the right choice.

    The field of service virtualization has rapidly become mature in recent years, going through the same maturing process we’ve seen happen to test automation in the past. That means that it’s time to take the implementation of SV seriously as well. With software development becoming more distributed, more complex and delivery cycles becoming ever shorter, you can’t afford not to at least consider SV seriously if you want to be able to keep up.

    Hopefully, this blog post has given you valuable information that can help you choose the best SV tool for your situation, no matter if that’s an open source or a commercially licensed tool.

    — Bas Dijkstra

  • Tricentis, QASymphony Merge: The Age of Continuous Testing Is Upon Us

    Tricentis, QASymphony Merge: The Age of Continuous Testing Is Upon Us

    Big news in the world of continuous testing today as continuous testing market leader Tricentis announced it is merging with QASymphony. Tricentis, a leader of continuous testing, is adding QASymphony’s leading testing automation platform to the mix. The full text of the press release is below.

    With the worldwide testing market pegged at more than $34 billion, this is a big move in a very big market. Tricentis raised $165 million last year from Insight Venture Partners and has put this money to good use: The QASymphony merger is Tricentis’ latest move. Previously, it acquired Q-Up, in June, and last year acquired Flood IO.

    Tricentis’ merger with QASymphony is happening at a time when continuous testing is becoming one of the hottest areas in DevOps. The demand for automated, continuous testing is driving a revolution in the QA market as companies recognize that quality is as important as speed when it comes to deploying new apps. It is a key cog in the modern software factory.

    I had a chance to sit down and speak with Tricentis CEO,  Sandeep Johri, about today’s announcement, the testing market and what the future may hold. Sandeep, who has previous experience at HP and other large providers, has a great vision of the market and where it is heading as well as the details that escape many “professional manager” CEOs. His hand is firmly on the wheel as he drives Tricentis forward in this space.

    Have a listen to our conversation below, and stay tuned—we will update and continue this breaking story as more details emerge.

     

    Tricentis and QASymphony Merge to Drive Innovation in Continuous Testing for Enterprise DevOps

    Next Gen platform disrupts the $34B testing services spend and transforms software testing for DevOps

    Mountain View, CA— June 20, 2018 — Tricentis and QASymphony announced their agreement to merge under the name of Tricentis. Tricentis’ market-leading Continuous Testing platform has earned the company unanimous “leader” recognition among the top industry analysts. QASymphony is recognized as the industry’s leading test management solution. The combined solution represents the most modern platform for transforming software testing to keep pace with digital transformation.

    “Tricentis and QASymphony are both recognized for their innovative testing solutions,” explained Sandeep Johri, CEO of Tricentis. “Tricentis is known for helping enterprises transform testing from a burdensome manual process that delays software delivery into an automated, continuous process that drives ‘quality at speed.’ The addition of QASymphony’s test management solution strengthens our leadership position and builds on our momentum replacing legacy testing tools.”

    “Today’s CIOs are responsible for ramping up innovation through accelerated software delivery,” explained Barry Libenson, Global Chief Information Officer at Experian. “Amid this demand for rapid innovation, CIOs are recognizing that software testing often delays their digital transformation goals. In fact, testing is now commonly cited as the #1 bottleneck to delivery speed. When testing lags behind development, the organization is forced to delay their innovation and that’s no longer an option. It’s inevitable that software testing must be transformed to keep pace with Agile development and DevOps.”

    “Global 2000 organizations remain locked into legacy testing platforms that were designed decades ago to enable manual testing processes,” explained Dave Keil, CEO of QASymphony. “Scaled agile and enterprise DevOps initiatives have changed the game. Yet, legacy testing technologies such as those by HPE (now Micro Focus) and CA are throttling speed and innovation. QASymphony has experienced triple-digit growth over the past year—allowing us to aggressively invest in our technology, infrastructure and people to meet the emerging needs of our customers.”

    QASymphony’s 100+ employees will be joining Tricentis’ 450+ employees. Sandeep Johri will lead the organization as CEO and Dave Keil will assume the role of COO. The combined company will have a global presence with offices in the US, EMEA and Asia. The transaction is expected to close within 90 days. EMBARGO UNTIL WEDNESDAY, JUNE 20, 2018 @ 8:00 AM ET USA

    The integration will reflect the commitment to innovation and customer success that both companies share—to learn more, register for the upcoming webinar. Additionally, the company will be showcasing the roadmap and solution at Accelerate 2018 in Vienna.

    DevOps Impact

    The market for software testing (recently estimated at $34 billion)1 has reached an inflection point. DevOps adoption is rising rapidly, and both Gartner and Forrester report that transforming testing is a primary factor for DevOps success.2 Yet, software testing remains a predominantly manual activity that delays releases and consumes 30-40 percent of an organization’s application budget3,4 Tricentis Continuous Testing solutions accelerate delivery and free up significant budget that can be reallocated to underfunded innovation initiatives.

    Gartner’s Joachim Herschmann and Thomas Murphy report: “As DevOps principles gain significant traction in enterprises, testing and QA is no longer a stage in delivery but an integral DevOps activity that exists in various forms through all areas. Automation has become an indispensable element in these efforts, and there has been renewed focus on software test automation tools as part of a larger toolchain that enables DevOps.”5

    About Tricentis

    With the industry’s No. 1 Continuous Testing platform, Tricentis is recognized for reinventing software testing for DevOps. Through risk-based testing, scriptless end-to-end test automation, and the industry’s most extensive technology support, Tricentis breaks through the barriers experienced with conventional software testing methods. Our innovative technologies simplify testing for even the most complex enterprise applications—transforming testing from a roadblock to a catalyst for innovation. The result is accelerated software delivery speed, improved cost efficiency, and reduced business risk.

    Tricentis is the only vendor to achieve “leader” status in all three top analyst reports (i.e., the “Triple Crown.”) This honor is based on our technical leadership, innovation, and a Global 2000 customer base of 800+ companies, including global enterprises such as Allianz, ANZ Bank, Experian, First Data, HSBC, Lexmark, Merck, Sentry Insurance, Starbucks, Telstra, UBS, Vantiv (now WorldPay), Vodafone, and Whole Foods. These customers rely on Tricentis to achieve and sustain test automation rates of over 90 percent—increasing risk coverage while accelerating testing to keep pace with Agile and DevOps. EMBARGO UNTIL WEDNESDAY, JUNE 20, 2018 @ 8:00 AM ET USA

    Tricentis has a global presence in Austria, Australia, Belgium, Denmark, Germany, India, Netherlands, Singapore, Switzerland, Poland, United States and the UK. To learn more, visit https://www.tricentis.com or follow us on LinkedIn, Twitter, and Facebook.

    About QASymphony

    QASymphony accelerates digital transformation with software testing solutions that help enterprises prioritize quality, develop reliable software and increase speed to market. The qTest platform streamlines software testing in agile and DevOps environments and centralizes testing efforts across the enterprise. By taking a strategic approach to quality, QASymphony enables enterprises to develop higher-quality software, faster, to better support new business models and customer experiences.

    The qTest platform streamlines development workflows by seamlessly integrating testing with leading automation, development and delivery tools including Atlassian’s Jira, CA Agile Central, Github, Selenium, Cucumber, Jenkins and many more. QASymphony is trusted by leading enterprises including Forcepoint, PFSweb, Cisco, Samsung, Verizon and Office Depot.

    1 Nelson Hall, https://research.nelson-hall.com/sourcing-expertise/it-services/software-testing/

    2 Gartner, Build Continuous Quality Into Your DevOps Toolchains: https://www.gartner.com/doc/3868367/build-continuous-quality-devops-toolchains

    3 Capgemini, HPE, Sogeti, World Quality Report: https://www.capgemini.com/service/world-quality-report-2017-18/

    4 Capgemini, HPE, Sogeti, World Quality Report: https://www.capgemini.com/service/world-quality-report-2017-18/

    5 Gartner, Magic Quadrant for Software Test Automation: https://www.gartner.com/doc/3830082/magic-quadrant-software-test-automation

    Media Contact

    Victoria Brown, Revere

    +1 (650) 850-2009

    Victoria.brown@revereagency.co

    QASymphony Contact Information Kira Mondrus

    QASymphony +1 (844) 798-4386 Kiramondrus@qasymphony.com

    Tricentis Contact Information

    Noel Wurst, Tricentis +1 (904) 673-4427 n.wurst@tricentis.com

    — Alan Shimel

  • Continuous Testing Insights from 2018 DevTest Research

    Continuous Testing Insights from 2018 DevTest Research

    The year is far from over, but there already several interesting DevTest surveys worth your attention. These studies don’t just quantify the obvious; they actually report some unexpected findings regarding how far and how fast we’re advancing, and offer some very specific advice on what’s needed to improve.

    We strongly recommend that you spend some time reading all three of these surveys in their entirety. However, in case you’re short on time (or impatient … or both), we wanted to highlight the findings that are most pertinent for readers practicing or researching Continuous Testing.

    Sauce Labs – Testing Trends for 2018: A Survey of Development and Testing Professionals

    [Read the complete report]

    2018 marks the fourth annual “Testing Trends” report, which is based on a global survey of more than 1,000 technology professionals responsible for developing and testing web and mobile applications.

    Key findings in terms of testing include:

    • 87 percent report that management supports test automation initiatives.
    • 45 percent expect to increase spending on test automation in 2018 (55 percent at large companies).
    • The number of respondents with high levels of test automation dropped to 28 percent in 2018 from 32 percent in 2017 .
    • The release cadence is actually slowing, with hourly deployments dropping to 5 percent from 14 percent and daily deployments dropping to 27 percent from 34 percent.

    In other words, everyone recognizes the value of test automation and most companies are willing to invest in it. However, test automation rates are actually decreasing, while Agile and DevOps adoption are steadily increasing. In the 2017 report, test automation rates increased slightly, and delivery speed also increased slightly. The 2018 reported a similar correlation: Test automation rates decreased, and the release cadence slowed down.

    GitLab – 2018 Global Developer Report

    [Read the complete report]

    This expansive survey polled 5,296 software professionals from around the world. The majority of respondents were software developers or engineers who worked for small- to medium-sized businesses (SMB) in the hardware, services and SaaS industries.

    Testing wasn’t a common topic in this development-focused research, but it did earn a prominent spot in the report. Testing was the No. 1 response to the question, “Where in the development process do you encounter the most delays?” A dubious honor—but not a surprising one. Last year’s DevOps Review polled an entirely different audience and came up with the exact same finding.

    VersionOne – 12th Annual State of Agile Report

    [Read the complete report]

    The 12th edition of the world’s longest-running Agile study found that while 97 percent of the 1,492 respondents’ organizations are practicing Agile, 84 percent report that their Agile adoption is not yet mature.

    Respondents feel strongly that two testing-related items would help them increase process maturity across both Agile and DevOps:

    • 83 percent want end-to-end traceability from business initiative through development, test and deployment.
    • 82 percent want better identification and measurement of risk prior to deployment.

    Respondents also reported a relatively high level of adoption of development testing and “shift left” testing techniques. Adoption levels were reported at:

    • Unit testing – 75 percent.
    • Coding standards – 64 percent.
    • Pair programming – 36 percent.
    • TDD – 35 percent.
    • BDD – 17 percent.

    Testers might also be interested in the survey’s feedback on Agile management tools. Usage rates were reported at:

    • Atlassian Jira – 58 percent.
    • VersionOne – 20 percent.
    • Microsoft TFS – 21 percent.
    • HP (now Micro Focus) Quality Center / ALM – 14 percent.

    The most highly recommended tools were VersionOne, Jira and CA Agile Central. HP Agile Manager, Hansoft and HP Quality Center /ALM were the least likely to be recommended.

    — Cynthia Dunlop

  • Why You Need Chaos Engineered into Your Hybrid Cloud Infrastructure

    Why You Need Chaos Engineered into Your Hybrid Cloud Infrastructure

    Nearly a decade ago, I began working at Amazon as an engineer. Cloud technology was just starting to revolutionize how companies launch applications, and early adopters were seeing the benefits of not having to purchase, provision and manage their own hardware. On the flip side, larger companies (especially those that store sensitive information) were still hesitant to fully embrace the black box that was “the cloud,” holding onto the belief that for critical applications, the cloud was neither reliable nor secure.

    Fast forward to today, and all of that has changed. We are seeing government institutions, banks, and hospitals all moving their infrastructure over to the cloud. The benefits of being agile, innovative, and highly efficient have mostly overshadowed the concerns around the cloud’s reliability. According to Cloud Technology Partners, cloud adopters see an average reduction of total cost of ownership (TCO) of around 40 percent. And top-performing banks are 88 percent more likely to implement a hybrid cloud solution than their underperforming peers, cites a recent report from IBM Institute for Business Value.

    Long story short, the question is no longer “if” but “which”—and there are three major options: AWS, GCP and Azure. Today, many companies are doing their best to avoid vendor lock-in, opting instead for a distributed strategy, where teams are empowered to choose the best option for what they are working on. While having a hybrid approach helps avoid vendor lock-in, it also can save teams from disaster if one of the cloud providers has a major outage like the great S3 outage of 2017.

    But just because you have a hybrid strategy—running active-passive across different cloud providers—doesn’t mean you’re home-free. Unless you’ve actually tested that your passive stack will jump in to save the day in the event of a crisis, you’re more or less closing your eyes and crossing your fingers in the hopes that your active stack never goes down. Think of it like running a fire drill: How can you be sure everything runs smoothly during a time of disaster if you never actually run the drill?

    We need to shift how we think about operations to place more of the burden of Ops work upfront. While it’s important to reduce MTTR (mean time to resolution), that still means there has been an incident that has affected customers and potentially your bottom line. As someone who has been a call-leader for Amazon and Salesforce, I know firsthand that we could save ourselves a lot of headaches by being more proactive about running experiments in a controlled environment and focusing on reducing the MTBF (mean time between failures).

    This is why it is important to have a chaos engineering solution that can run experiments across all of your environments regardless of their build, technology or provider in one simple to use, unified experience. By consistently testing that your “failsafe” actually behaves the way you want it to, you can be much more confident that the operations work you’ve put in to make your architecture more resilient isn’t all for naught when a real-world outage comes calling—and it will.

    — Matthew Fornaciari

  • DevOps Chat: The Kitty Hawk Venture & Continuous Testing w/ Aruna Ravichandran

    DevOps Chat: The Kitty Hawk Venture & Continuous Testing w/ Aruna Ravichandran

    A great new book on DevOps is coming out called “The Kitty Hawk Venture.” In the vein of “The Phoenix Project,” it is a novel about one fictional company’s journey specifically with continuous testing. Based on a real-life airline company, the book is great case study and learning experience around continuous testing and its importance to the entire CI/CD and DevOps experience.

    The book is written by Jeff Schaeffer, general manager of the Continuous Testing business unit at CA Technologies; Alex Martin, CTO of Continuous Testing at CA Technologies (and former CTO of Rally Software); and Aruna Ravichandran, vice president of  Gobal Marketing at CA Technologies and author of “DevOps for Digital Leaders.” Beyond the book, there is a companion ebook on continuous testing, as well as other related content.

    In this DevOps Chat, we speak with Aruna, who is a frequent guest on our chats, about the book, continuous testing and the overall DevOps market. It is an excellent chat and, as usual, Aruna’s passion for what she does and DevOps shines through.

    For those not familiar with our DevOps Chats, the streaming audio of our conversation is immediately below, followed by the transcript of our conversation.

    Audio

    Transcript

    Alan Shimel: Hey everyone, it’s Alan Shimel. devopsy.kinsta.cloud. You’re listening to another DevOps chat. This is gonna be a really great chat. I have one of my dear friends in the DevOps industry joining us. Aruna Ravichandran Ravichandran, VP of Marketing DevOps at CA technologies. But more than that, now a multi-time author as well as a code engineer and a bunch of other things. She’s had an amazing, amazing life and career. Aruna Ravichandran, welcome.

    Aruna Ravichandran: Thank you, Alan. I am very excited to be here with you today talking about my latest book.

    Shimel: Yup. So it’s a new one coming out, hopefully in July I guess is the current timeline. And it’s called, “The Kitty Hawk Venture.”

    Ravichandran: That is exactly right. So you know, I think we chatted about a year ago when I wrote my first DevOps book. It was called, “DevOps for Digital Leaders.” And what we decided to do based upon the substance of the book is to basically to take a leaf from the Phoenix Project and write this book in terms of a real life story about one of our customers. I can’t name who the customer is. And the book is actually called, “The Kitty Hawk Venture.”

    And it’s a novel about continuous testing in the world of DevOps in order to help customers as they’re going through their digital transformation journey. So it’s not written as a continuous testing handbook.

    But it’s basically a real life example which happened with one of our customers and tells the story of the journey they actually went through in terms of how they actually transformed their testing discipline, lines of automation down to continuous testing and how they actually addressed some of the bigger challenges they actually had internally across the board.

    Shimel: Got it. So Aruna Ravichandran, I promised you and I promise our audience, no spoilers on this. So I don’t want to go too much into the actual story of the book, let’s keep it at a high level and say that it is about continuous testing and it is based on a real life customer experience. Maybe an amalgamation of customers.

    But you know, Aruna Ravichandran, you mentioned your first book, “DevOps for Digital Leaders.” So this is then your second book on DevOps.

    Ravichandran: Yes, it is. But this time I’ve actually focused on one of the emerging disciplines within DevOps. the first book was primarily focused around DevOps as a discipline, some of the challenges customers actually face.

    Where do they actually start. Because you know, there is no hard and fast rule in terms of DevOps. you can start with continuous integration. You could start with continuous operations. Or you could start with continuous testing.

    The book was primarily focused around end to end DevOps and gives some amount of coaching to the end reader on where to start, what’s the maturity model, what are some of the metrics, how do you measure the substance of DevOps.

    It also talks about several different customer examples on the journey they actually went through. But and speaking from that, we saw that a whole bunch of our readers are interested in learning more about an emerging discipline around continuous testing. So this book is primarily focused on continuous testing as a discipline and why it’s a very important area within the world of DevOps.

    Shimel: Yup. Now, Aruna , though, you hear it in your voice when you’re speaking to our audience here – it comes through loud and clear. this whole area of DevOps – continuous testing specifically with the book. It’s something that you are passionate about. I get it, you’re a VP in the DevOps group. You should feel an affinity. But this seems to go beyond your title and your job. It’s truly something you have a passion for.

    The book is almost 200 pages from the early view I saw. You don’t write 200 pages, you know, blah, blah, blah. You’ve gotta have sort of a burning passion. What is the passion that’s driving you?

    Ravichandran: So the passion that drives me is my customers. As I talk to my customers in the DevOps space, I am very passionate about solving customer problems. And as I talk to my customers, especially customers who want to make an impact, who are on their digital transformation journey, what I learned was testing is a discipline which has existed for several different decades across the board, the very first time when this actual code was actually implemented.

    But what you – what I actually found is that people have testing practices. But it’s mostly around islands of automation. Very, very few customers actually are doing end to end testing, which I call continuous testing.

    And even though you have islands of automation, what I actually find is that testing still continues to be manual. I did a poll of a whole bunch of enterprise customers across the board, and I was shocked to find that testing still appears to be 70% manual across the board.

    And given all of the – this is my personal reason for why I am very passionate about continuous testing, because I think that DevOps has gained a tremendous amount of precedent, but if you don’t embed continuous testing as a discipline across the software development life cycle, you really cannot go through DevOps.

    So continuous delivery has a lot of what do you say affiliation with the market. More and more people are getting educated on how to basically go through the CICD pipeline so that it can go from code to production in minutes. But what they realize is that if you don’t have the testing discipline embedded across the SCLC, going through the pipeline doesn’t buy you anything.

    And so that’s why this is a very important topic which I am very passionate about.

    Shimel: I mean, Aruna Ravichandran, on some level it’s almost – and I think I’ve heard you say this, continuous testing is almost the missing link. Right? In the continuous delivery, in the software development life cycle delivery chain. Right?

    For a lot of – you know what I mean? Companies, they focus on configuration management, on CICD, even on security. But that continuous testing is – it’s right there. It’s a lynchpin. The other thing I wanted to just quickly mention about it is continuous testing isn’t just a pre-delivery —

    Shimel: It’s truly continuous.

    Ravichandran: You nailed it. I would 100 percent agree with you on everything you stated. It is the missing link within DevOps. continuous testing is a practice of testing across every activity in the software development life cycle with the hope that you can uncover and fix unexpected behavior as soon as it’s ingested.

    So it’s about embedding testing as a fundamental and ongoing aspect of every activity through the application life cycle, going all the way from requirements down into production, while facilitating the pre-exact mechanisms what happens in the world of DevOps to where you can actually fix the problem throughout the life cycle.

    Shimel: Let me return to “The Kitty Hawk (Venture),” here again if I can for a moment. You mentioned earlier you took a page out of Gene Kim’s Phoenix Project book in that this isn’t similar to other books you’ve written and that I’ve even reviewed. That are your typical sort of business textbooks, right? You follow an outline and a table of contents and it goes. “Kitty Hawk” is more like you say – a novel, I think is the right word for it.

    How does that make it different for the reader? Or the listener if it’s an audiobook? For someone reading the book.

    Ravichandran: Yup. So for example – this book “[The] Kitty Hawk Venture” is actually based on the main character in the book is Leigh Freemark and she’s a senior director for quality assurance at a company called Runway Air. So the vertical we actually chose was an airline company.

    The way this book is actually written is it talks about the journey of what they actually went through from a transformation journey. So they experienced a major glitch in terms of quality and which caused a substantial amount of negative reputation for Runway Air.

    So this book actually walks through Leigh’s journey, the journey of her own personal mission of making quality first as a big part of her agenda.

    And the way this book is written – we have so many technical books which basically come into the market on DevOps and on testing, so on and so forth. So we wanted to basically – we took a leaf from “The Phoenix Project,” and we also took another leaf from “The DevOps Handbook,” which is the second book which was recently announced by Gene Kim.

    So this is basically written as a novel which tells you the journey which Leigh Freemark went through with Runway Air, but at the end of each chapter we give guidance in technical terms, of what was the actual continuous testing journey Leigh actually went through.

    Right? So that’s why I said, it’s written as a novel but at the end of each chapter we also have like a handbook around the continuous testing discipline on what changes people actually need to make in order to fully embrace continuous testing as a discipline in the book.

    Shimel: And even beyond that, Aruna Ravichandran, from what I understand and correct me if I’m wrong, you guys are planning on coming out with sort of an eBook companion to “Kitty Hawk” on “The Definitive Guide for Continuous Testing.” Is that correct?

    Ravichandran: That is exactly right and so we just actually published it and it will be out very soon. It’s called “The Definitive Guide to Continuous Testing.” And here, we walk our users through 11 different disciplines on continuous testing.

    Ongoing all the way on small teams [sound drops] _____ to how basically they can go through the transformation journey on continuous testing. And at a very high level, I walk you through eleven disciplines and you will see that completely in the novel because that was the journey which Leigh Freeman actually went through in the book, is the first discipline how do you basically and it will – the – you know, all of your testers.

    It could be a developer who is actually doing unit testing or it could be someone who is doing API testing or it could be load testing. Or it could be reproduction testing. All testing in production, you need to be able to have your environment set up. How do you quickly virtualize your environment becomes a very important part of the mix.

    How do you basically break on your environment, how do you actually create your environments very, very quickly is one of the first disciplines. The second discipline talks about testing management right? So regardless of what testing you’re doing having the right data at the right time is very critical. So how quickly can you generate your data?

    And given the world of GDPR right now, you know, which basically went into a test in May, people have to be even more critical about generating test data. You want to make sure that your GDP are compliant. And so having the test data at the right time is critical but also being compliant with a whole bunch of mandates out there is even more critical. The third discipline talks about test automation. How do you basically fully orchestrate the application pipeline.

    So that if the scripts actually fail, you can orchestrate how you can actually achieve continuous testing. The fourth one is around pipeline orchestration. This is the backbone. Everything is tied to it. everything must be integrated with your entire automation stream. You need to understand how it works, how to interpret the results and how do you make quick decisions in order to find out which part of testing is actually a bottleneck?

    And this is what we actually call pipeline authorization. The fifth one focuses on the various different competences, right? If you are testing, the world is going more towards an API economy. So how do you actually bring API testing discipline so that you reduce the reliance on UI testing? Especially for business logic testing.

    The sixth one is around performance and load testing. Right? So you can’t think about performance and load testing toward the end of the testing life cycle. You need to think about shifting left performance and load testing so that you can enable development teams to actually think about this much more earlier on.

    The seventh one is the around the new emerging practice around security testing. So this is where everybody talks about DevSecOps, it’s all about shift left security. How do you enable developers to actually embed static code analysis as a part of their unit testing?

    How do you basically get them to think about security testing by making it easy for them? By having it integrated as a part of their ID environments, whether it’s intelliga or it’s xless or visual studios, so on and so forth. So security testing is the seventh one.

    The eighth one is accidents of test driven development and behavior-driven development. So you take the accidents criteria for each user and create that to ensure that those were actually met. Right? So this keeps testing focused within sprints and ensures developers develop what the business actually expect.

    The ninth one is around automated test generation. And this is the activity of manually designing and writing manual tests of automated tests which is the biggest bottleneck.

    They are the computing magazine data surveyed very recently and they asked, which is the biggest bottleneck across your software development life cycle? 63% of the practitioners actually said that testing was a bottleneck. And why was testing a bottleneck? Because it takes time.

    Think about it, you have actually created 200 test cases for your regression suite for a vertical feature you are going to launch. Let’s assume you’ve caught a major defect and you fix the vertical defect, it means that you have to go back and recreate all of your regression text suites.

    And if there was an automated way that you could take your requirements and as things changed in production you can recreate your test suite, that will be a tremendous amount of savings for anybody to actually not have to create the test cases manually which then eliminates the bottleneck which we actually have.

    The tenth one is around requirement engineering. You must include all of SDLC stakeholders into your continuous testing journey. This means finding a better way to communicate and collaborate right from the requirements phase. The earlier one includes requirements engineering into continuous testing initiatives, the smoother that continuous testing journey will be. Because team members will be working with the same understanding right from the beginning.

    And the last but not least – the eleventh discipline talks about feedback loops. This is the heart and soul of DevOps, it still continues to be the heart and soul of continuous testing. These are critical to ensure that you can have successful continuous testing.

    Shimel: Excellent. You know, Aruna Ravichandran, sitting here looking – I want you to talk about quality, I don’t know if we have time. Maybe we’ll do it on another show. But the 11 disciplines, you did a great job laying them out here. I’m just wondering if someone listening says to themselves, oh my goodness, where do I begin? This is daunting. There is eleven of them. What is your advice to someone just starting out, listening in here – do I do eleven at all at the same time? What do I start with? What – do I take the path of least resistance? How do I get started with this?

    Ravichandran: So I break it up into six different areas, so this is a question you get asked for DevOps and you get asked the same question around continuous testing. So if I have to give guidance on techniques on how to actually adopt – it’s also a cultural transformation too.

    So the first one is, you know, you need to figure out a way to improve the relationship between the tester and each developer. So keeping the team small and creating inter-team collaboration makes it very easy to actually report and actually share the results between the tester and developer.

    So that would be one important tactic which needs to be adopted from a cultural transformation perspective. Number two, is like I said, people actually have automation within testing, which is what we call test automation. But they really haven’t automated the whole thing. So I would say continue to make automation a priority.

    So figure out where people are spending a lot of their time – it could be around building a regression test suite, because they tend to be manual. So how do you automate? So keeping that automation mindset is important. Figuring out where you’re spending cycles of your time in terms of doing manual tasks is important. So making automation a priority is important.

    Map out within your SDLC and identify your automation opportunity. Number three is make it small – right? Like get small. Break it down into smaller increments so that it’s easier to test after every aspect of the SDLC. So break down increments after coding. Break it after design. This makes it easier to automate the test and it also makes it much more easy to deploy and integrate as a part of continuous delivery.

    The fourth one is you want to keep track of everything. Use metrics. Start-up criteria. Right? So your defect density rate is high, figure out how can you reduce the defect density rate. Continuous testing is all about immediately identifying if things are working or not. How do you basically rinse and repeat if it’s working, or how do you actually pivot and change if it’s actually not working.

    The fifth one is getting the right continuous testing tools. Tools play a big role. So we talked about also transformation, we talked about processes and practices, and also having the right tools is important. Figure out the tools that help you develop tests and analyze continuously.

    Pick the best of breed tools that work together in a way that incorporates very easily into your continuous delivery environment. Choose tools which have community based support, that have open source plans that everyone shares the challenges and solutions and have interesting and lead to interesting use cases.

    And last but not least, the number sixth one, is develop a system to display your results. So do deep dives into the results, because that shows you that if you have your code not working, where the gaps are. define your KPI and your acceptance criteria and make them quantifiable. Create dashboards to track those KPIs, including a baseline so that you can actually track with the sequence changes. So these are some of the disciplines I always share with my customers as well as my clients.

    Shimel: Excellent. And now with everyone listening on here. Aruna Ravichandran we’re almost out of time, one more question if you have a second.

    Ravichandran: Absolutely. I do.

    Shimel: So I don’t know if we’re letting the cat out of the bag. If you can’t talk about it, politely say so. But you’re about to roll out something we hear called continuous testing academy? Launching this summer?

    Ravichandran: Yup.

    Shimel: So what’s that about?

    Ravichandran: So the key thing – going back to the question you asked, this continuous testing is like an emerging discipline. People want to know what do I need to do, how do I need to start? How do I get there. And so last year I launched the – I saw that there was a tremendous amount of need in the market around JMeter.

    And so I launched JMeter academy in order to basically to provide input to the community so they could actually train themselves on JMeter. But this year, knowing that continuous testing is gaining tremendous amount of momentum, I’m going to launch a continuous testing academy.

    This is going to have a basic module, it will also have an advanced module. And people will actually have the opportunity to go through the entire module and get themselves certified. We’re also going to launch a certification and you will have the opportunity to become continuous testing certified by DevOps institute. So this is our way to give back to the community in order to basically get people understanding on how they need to evolve their continuous testing practice in order to make a bigger impact in the DevOps world.

    Shimel: Excellent, again, Aruna Ravichandran – great job. You are just, I mean, literally, what a dynamo. All over the place. Doing so many different projects. All with DevOps and continuous testing as a theme there. Congratulations and hats off to you. One of my heroes. Anyway, we’re about out of time. Thank you so much, Aruna Ravichandran, for giving us a preview. Once Kitty Hawk is available to the public, I’d love to have you back on and perhaps we can chat more about it, sound good?

    Ravichandran: Yep. That sounds awesome. As always, Alan, it was excellent talking to you. I love your energy, I love your passion. And looking forward to partnering with you more as we basically enable our customers and prospects to continue on marching further along in the world of DevOps.

    Shimel: Well stay tuned for some announcements, we’re working on it. Aruna Ravichandran, I will be in touch. Aruna Ravichandran Ravichandran, a VP product marketing, marketing DevOps, CA technology, multiple times author, and really DevOps enthusiast. Thank you for being our guest on DevOps chats today. We’ll speak to you soon. This is Alan Shimel for devopsy.kinsta.cloud. You’ve just listened to another DevOps chat. Have a great day everyone.

    — Alan Shimel