devopsy.kinsta.cloud

Author: Noel Wurst

  • Continuous Testing at More than 200 mph

    Continuous Testing at More than 200 mph

    “The old principle of tinkering with an instinctively designed car has long since been superseded by systematic testing of every major component and structure – both before and after the car is fully built and ready to race.”

    —“Understanding F1 Racing,” Formula1.com

    While speed is the most obvious shared goal between racing and software development teams, you might be surprised at the striking similarities between how these teams achieve that goal. Each relies on top talent, funding and innovation to reach the speeds at which their competition cannot, but there’s another oft-ignored component that’s quickly becoming one of the greatest differentiators between first place and everyone else: testing.

    The way F1 teams test their cars has gone through a transformation just as monumental as what we’ve seen in the software world over the last decade. The ability to continually test every major component and structure of a race car or piece of software is crucial, but the ability to quickly respond to the data that comes out of those tests is just as important. It’s continuous testing before, during and after the rubber meets the road—or the software hits the marketplace or app store—that determines the winners.

    The Evolution of Testing

    In the aforementioned Formula1.com article, an all-too-familiar scenario to those in the software world is described:

    “By midway through the first decade of the 21st century, a typical Formula 1 testing programme had become a major exercise in both manpower and logistics…spiralling costs, which invariably resulted in development work that was invisible and meaningless to fans …”

    Replace “Formula 1” with “software” and the above excerpt is just as factual for those who don’t know the struggles of yesterday’s approaches to software quality, and just as painful for those who personally experienced them. Manual processes, insufficient test automation technologies, shared unrealistic environments, lack of trust between teams and out-of-control costs are constraints of the past for today’s top-performing teams, but they remain for many still relying on legacy testing approaches and technologies.

    What Continuous Testing Looks Like Today

    Some of the racing enthusiasts from Tricentis’ leadership team were invited earlier this year to meet the Haas F1 racing team during their 2018 preseason testing at Circuit de Barcelona-Catalunya. We witnessed firsthand the remarkable transformation that Haas and F1 racing as a whole have embraced. Track testing is where, as Formula1.com says, “the steady evolution that happens to all Formula One cars during the course of their life begins, a constant improvement of tiny details and setup.” This constant evolution and iterative, rapid innovation is exactly the goal of high-performing software organizations today.

    After spending the day at the track, we noted the following approaches and methodologies that we see in innovative continuous testing and DevOps practices today:

    • F1 cars are continuously delivering data and feedback to the organization.
    • The cars have millions of data points and thousands of sensors, providing actionable data.
    • These data points and sensors are grouped into core processes, and isolated by key questions.
    • Test runs are done with the cars in as realistic environments as possible.
    • The speed at which that data can be received, interpreted—and most importantly—acted upon, is the difference between high-performing and low-performing teams.
    • The tiniest changes to the car make enormous differences. Nobody is showing up in an entirely different car, they’re making minute changes.
    • These cars cannot fail on race day or the race is lost.

    Today’s high-performing organizations are treating the delivery of software in the exact same manner as the world’s leading racing teams. Innovative technologies are being put into each stage of the delivery life cycle that provide invaluable feedback that helps enterprises better understand risks to their business. Being able to rapidly analyze—and act on—this amount of data has become imperative for avoiding disruption. Testing software in environments that are as realistic and as close to production as possible is mandatory, and those environments need to be available when they’re needed most. These needs have driven the adoption of what’s known as continuous testing—an evolution of traditional test automation that has proven unable to support software’s increased complexity, and expectations around the pace of development and delivery.

    What Questions Are F1 Racing and Software Enterprises Asking?

    While each have their own unique challenges, there are many similar questions that need answers—in real-time, and at any development, testing, or release stage—that the racing and software industries share.

    • How fast are we today and how fast will we be tomorrow?
    • How fast is our competition, and how fast will they become?
    • What are we measuring today that’s impacting the business?
    • How quickly can we introduce change?
    • What data are we using to validate that change?
    • What’s the risk of not changing?
    • How low can we get our mean-time-to-repair?

    The fastest time to market—or time to the finish line—is how you win the race. Your developers or engineers don’t just build the fastest car, or the most popular app to today’s standards and then call it a day. They’re constantly looking at what speed, UX, functionality or accessibility will be needed to be the best tomorrow, and then iteratively adding those improvements as quickly as possible. They don’t stop engineering, ever. And testing shouldn’t be performed any differently. Each new feature, condition, region, device, persona, currency, language and OS that your app needs to offer, integrate with, accept or run on is going to need continuous testing before, during and after that app’s release.

    Whether you’re responsible for the quality and performance of a billion-dollar race car or the applications for a billion-dollar enterprise, speed and quality are your path to the winner’s circle. Continuous testing ensures that you remain there.

    — Noel Wurst

  • Tricentis Accelerate Conference’s Big Reveal Came from its Audience

    Tricentis Accelerate Conference’s Big Reveal Came from its Audience

    At the recent Accelerate San Francisco 2018 conference, Tricentis was joined on stage by big-name customers, global partners and well-known quality and DevOps evangelists. This was the first time a U.S. date for Accelerate was added alongside its annual stop in Vienna, Austria. And, while the number of attendees was impressive—especially when Google IO and Microsoft Build were held the same week—it was the undeniable sense of urgency running through the audience that stood out, to me, as the week’s biggest takeaway.

    Having covered countless software and technology conferences over the years, it’s always somewhat eye-rolling when the hosting vendor or conference organizer uses their time on stage poorly. I often see this done in one of two ways. They either, a) relentlessly try to sell you on a current transformation or shift that no one really buys into, or, b) only explain that there is a shift—one that everyone agrees on—but then offer no actionable information for those who will be impacted by it.

    Agile, DevOps, digital transformation, continuous integration/delivery—all of which came up throughout Accelerate—have been the focal points of software conferences for years. These shows have given development and operations teams a wealth of information, case studies, lessons learned, best practices and inspiring keynotes, and they should be commended for it. But, today’s CIOs are frantically looking for the culprit behind how they’ve invested so much to make these processes and methodologies possible—yet, are still struggling to get high-quality, software out the door faster. Many of these CIOs are realizing the glaring oversight that they, and these conferences, have made.

    Accelerate San Francisco’s manual testers, automation engineers, exploratory testers, test managers, QA directors and more didn’t have to be sold on the idea that testing has an enormous opportunity to solve the challenges their CIOs are facing today. Those who truly respect the power of “good” testing (an the artistry behind it) have always known what testers were capable of providing to the business. However, testing has been hamstrung by legacy technologies that prevented moving at the same pace of their development and operations colleagues.

    But, today, those constraints have been removed, thanks to innovative test automation technologies and testers and QA professionals may not even have to sell their “new” worth to management. More and more often, the C-suite is recognizing software testing as an untapped resource for innovation, and many of Accelerate’s attendees could clearly sense the attention and expectations from their leadership on the horizon.

    Day 1 of Accelerate was largely product-focused and featured a number of “discovery” and “deep dive” sessions led by the product owners of some of Tricentis Tosca’s most popular functionality. Continuous Testing, Load Testing, API Testing & Service Virtualization, SAP Testing were all showcased and demoed to existing customers and those evaluating whether Tricentis could assist them on their own unique journeys. Day 1 wrapped up with a product road map discussion given to a standing room-only crowd, and led by Tricentis chief product officer, Dr. Gerd Weishaar.

    Day 2 kicked off with a bang with what was dubbed, “The Great Debate in Software Development.” Electric Cloud’s Anders Wallgren, Accenture’s Jeff Wilkinson and Tricentis founder Wolfgang Platz sat down to discuss who owns quality today, why that responsibility might change, and how testers and SDETs can be prepared for the future. There were some really outstanding—and oftentimes, conflicting—points made by each panelist, including:

    “Development is a skillset; quality engineering is a mindset.”

    “Testing is always reactive; quality engineering is proactive.”

    “Quality is a frequency and depth of measurement game.”

    Once we get the debate’s recording online, I’ll share the link in a future blog, and would love to keep the conversation going with anyone who is interested. There was so much that was presented that’s worth discussing and debating further.

    Dr. Nicole Forsgren took the stage immediately after the debate and introduced her new book, “Accelerate,” (how could we not have invited her to a conference with the same name?) that she co-authored with fellow DevOps household names Gene Kim and Jez Humble. I’m constantly looking for ways to get testing more on the minds of DevOps community (because it makes zero sense for it to be so absent), so I was thrilled to learn that continuous testing has made it into DORA’s next “State of DevOps” report. Their annual survey that this report is focused on should be released soon, so look out for that.

    Tricentis CEO Sandeep Johri then took the stage to give his thoughts on Tricentis’ role within the current state of software testing. Johri pointed out that while software becomes the key to success in many digital transformation initiatives, it also becomes a serious problem when software cannot be released at the speed that business demands. “Who cares if you can develop faster if you can’t also ship faster?” said, Johri. “You cannot ‘ship and pray’ in the enterprise.”

    Being able to ship faster—with a sound, risk-based assessment of each release candidate—requires continuous testing, said Johri. And, to CIOs who historically have viewed testing as a bottleneck to innovation, I can see where this could be a hard sell. “You know that thing that you hate? The thing that costs a fortune, and takes forever, and doesn’t provide you with an easily discernible ROI? Do that way more, or, better yet, do it all the time.”

    But, as Johri remarked—and I’d never actually heard someone word it this way—”Testing isn’t a bottleneck. Scripting is.” This is 100 percent accurate, and, as CIOs in greater number begin to realize this, continuous testing will become not just better understood, but a must-have in every organization. It’s the manual effort around testing that slows you down, stalls innovation and adds increased costs around the requirement for testers or developers to have coding expertise. These costs and bottlenecks are what CIOs want to eliminate. As awareness of modern test automation technologies that remove these constraints increases, Johri predicts that the solutions that can’t offer the same level of agility at scale will continue to fade from the market.

    The next keynote of the day was given by Tricentis founder Wolfgang Platz. While Platz gave due-credit to what I’m calling the “A.I.-lephant in the room,” it was the advice to the Accelerate attendees of how to navigate the current transformation going on in world around them that I found most inspiring. Platz shared the current strategic priorities for CIOs in 2018, and it was remarkable how clear it was to everyone in the crowd where they could assist with meeting each one of them.

    We hear all the time about lofty aspirations such as, “Quality is everyone’s responsibility,” and, “DevOps establishes that every team is working toward the same goal.” But, when Platz ran through CIO priorities such as digital transformation, customer experience, data analytics, moving faster, improving culture, scaling agile development and freeing up budget, eyes didn’t roll. People didn’t groan or walk out. It was clear that the entire room understood exactly what they were capable of by trading the limitations of yesterday for the capabilities of tomorrow.

    Today’s CIOs have never been under more pressure to avoid disruption, and my biggest takeaway from Accelerate was that the testing community understands that accomplishing this monumental task—in every software release—is only possible with their involvement.

    Let’s see how many testers continue to navigate through change introduced by others, and how many take the opportunity to drive new change themselves.

    — Noel Wurst