devopsy.kinsta.cloud

Author: Renaud Boutet

  • Why Browser and API Testing is Shifting Left

    Why Browser and API Testing is Shifting Left

    For developers working with cloud-native technologies, software testing and quality assurance is a key part of product development. For most teams, testing has “shifted left,” so the quality of the code can be assessed as it moves through the CI/CD pipeline. This approach to quality assurance reflects the rise of DevOps and other modern development practices – developers today own their applications from end-to-end, and conducting testing early in the product life cycle is how they find potential bugs, save their organizations time and money and build products that delight their customers.

    More recently, developers have added end-to-end API and browser tests into their CI/CD practices, extending the scope of their testing and quality assurance practices. By testing how code performs on different browsers and how it interacts with important data sources, developers have added another layer of quality testing to the early stages of their development processes. This has brought added benefits to other “shift left” practices, such as unit testing and security testing, and has led to a new understanding of quality assurance.

    Traditional Approaches to QA Can’t Support the Pace of the Cloud

    As companies have migrated to the cloud and adopted agile development methodologies that align with DevOps practices, they’ve encountered conflicting priorities between releasing code quickly and maintaining the highest level of quality on each release. Traditional manual testing techniques were not built for agile workflows, but instead supported older waterfall methodologies, in which testing could be conducted as a discreet instance before code went to production. With agile development, having a specific team in charge of all testing right before production is not the most sustainable approach, as it hinders the rapid pace of development that the cloud enables. This is why testing is “shifting left” to earlier stages of the development process, and why manual testing is being done by people embedded within each engineering team.

    Cloud-Native Teams Have Shifted Left Into CI/CD Pipelines

    As part of these agile methodologies, companies have moved toward leveraging CI/CD practices to ensure code is always in a state where it’s ready to be deployed. As code moves through these pipelines, developers want to ensure they are able to catch bugs earlier in the development cycle. To do this, they’ve taken to testing earlier and more often.

    In particular, developers have increasingly recognized the value of end-to-end testing throughout their CI builds. API tests allow teams to verify end-to-end workflows by chaining HTTP requests and API calls, so they can validate all the layers of their systems from worldwide locations. Browser tests capture the user’s view during critical transactions, so teams can understand how UI changes are impacting web applications. Shifting both left into CI/CD pipelines saves time and money, as fixing a bug earlier in the development lifecycle is much less expensive than trying to fix it at the last minute, when it has grown in complexity and scope.

    It also means many issues are caught by the developers, without the intervention of another engineer manually validating their work. This reduces the number of cycles required to find a bug, allowing the code to be circled back to developers, fixed and re-deployed to a testing environment. When regressions are caught early, developers can focus on the details that will make the product great.

    Visibility of Metrics, Traces and Logs Drives Value

    Running browser and API tests in your CI pipelines is beneficial, especially when teams can correlate these tests with data from across their stack – front end and back end, application and infrastructure. With correlated metrics, traces and logs that can be accessed easily without toggling between different products, teams will have the full context necessary for both detecting and troubleshooting failed tests. Detecting the failed tests allows for the rollback of bad code before it goes to production, and the correlated full stack data can be used to troubleshoot and fix any issues not caught during the development stage. This creates a new culture of quality assurance, in which maintaining quality is a holistic process that involves every part of the development cycle, and draws on data from every part of the stack.

    Consistent Quality Requires Collaboration

    This change in quality assurance goes beyond the practice of shifting browser and API testing left, and represents a fundamental change in how engineering teams collaborate within an organization. Testing happens before code goes to production, continues with robust monitoring once code is live, involves data from across the stack and is the responsibility of teams from various parts of the business. With this collaborative approach, for example, a front end engineer can easily identify an issue in the integration of their latest changes with the way the back end APIs are structured. They can then engage with the back end team owning this service to collaboratively iterate and find the best solution to resolve the issue. It may start with a shift left, but, ideally, it ends with the removal of silos and the opening of communication channels among teams.

    Automated QA Delivers Customer Value Faster

    A fully operational approach to quality assurance means that issues are resolved faster, or even resolved while code is still in development, so they never become issues at all. This creates a virtuous cycle – fewer regressions means less downtime, which makes customers happy, and frees up engineers to build new products, which makes customers happier still. That baseline of reliability and customer experience enabled by effective quality assurance is what allows for new innovation that delivers value to customers.

  • How to Automate Testing for Optimal Customer Experience

    How to Automate Testing for Optimal Customer Experience

    During a sudden surge in online traffic like the one many companies have seen over the past few months, automated testing can help organizations optimize the customer experience

    Imagine you manage an e-commerce office supply company and are dealing with a sudden surge in traffic—with more people working from home, demand for your products has grown significantly. But when a customer attempts to complete a purchase, they encounter a debilitating error such as a broken button or a page that won’t load. Now imagine this happening to hundreds of customers simultaneously, during the most critical time of day for purchases on your site. Imagine the revenue lost, the customers never to return, the complaints on social media—it’s a situation everyone wants to avoid.

    But this is a situation many will have to handle at some point. Web pages are large, complex beasts made up of different images, fonts, style sheets, local JavaScript code, asynchronous requests to the backend and more. It’s easy to forget how many scripts hold a website together. And any small change can have a butterfly effect on the overall page performance.

    What does the user actually see and experience on the website? And what happens if changes are made in the backend? Does the “Buy” button suddenly disappear? Is everything still working as it should? To understand how an application behaves with users, companies need to understand those users’ experiences when they interact with the application over a browser. All too often this means manually checking the availability and functionality of individual online applications, and then hoping that users do not stumble upon additional problems that may have fallen through the cracks.

    Accounting for and identifying these potential problems and regressions is not only an uncomfortable process, but also a lengthy one. Especially in times of increasing traffic and new digital business, as we’ve experienced in these times of WFH and e-commerce, the continuous review of all digital experiences is essential. A customer’s first impression is crucial, and even a long-term customer may go elsewhere if a page doesn’t load or if a button doesn’t work. Options abound on the internet—you can’t risk someone giving up on your business and trying a competitor’s.

    Why Automating Browser Tests Is Important

    In browser-based applications and websites, browser tests can monitor the user experience and ensure that user actions, such as registering for a new account or adding items to a shopping cart, work smoothly—even after changes in the backend.

    Popular browser test solutions have long since reduced the manual effort required to confirm that new features didn’t break already existing functions, but the implementation and maintenance of these tests is very time-consuming. In addition to learning a new test framework for these solutions, teams need to set up a test infrastructure and update faulty tests with every change to the user interface, costing valuable time that could be used for new product development.

    The latest generation of fully automated browser tests, on the other hand, allows everyone on the team to record and automate rich browser-based test scenarios in minutes—without any code. The “Tester” opens the corresponding website or application via the test tool and starts the recording. As she interacts with the website or application, the solution automatically records all actions as a series of steps, which can then be edited or changed when reviewing the recording. Assertions can also be added, such as whether a user sees a welcome message when they log in.

    If a test is recorded, it is possible to define how often and on which devices it is to be carried out and from which places around the globe the tests are to be played to mimic users’ behavior. The recordings can be played over a variety of websites, including internal, non-publicly accessible applications or pre-production and CI environments. For each test run performed, the end-to-end duration of the test and a visualization of the duration of each step are provided. A screenshot is automatically created for each step and each query so testers can follow exactly what the user sees in each respective step. All the frontend errors and resources happening along the test are also collected for a simplified troubleshooting of failing test runs.

    Because websites are constantly modified by developers, once a test is created, machine learning algorithms detect changes to the application and automatically update the tests accordingly. If a browser test fails due to frontend or backend problems, an automated tool provides the context for quick troubleshooting. In addition to screenshots, frontend errors and frontend resources, the behavior of the backend services and infrastructure for this specific step is also provided.

    Using Time More Effectively

    Automated browser tests have traditionally been notoriously fragile, often breaking with minor changes to the user interface. To eliminate false positives, today’s ML-supported browser tests mimic human decision-making processes and adapt intelligently to changes in the application.

    For example, if a button is moved in a web application or the identifier of an element changes, the browser test derives from this how the existing test should be carried out in the updated user interface. If all steps and assertions in the test are successfully completed, the saved identification mechanism and test definition will be updated to reflect the changes on the page.

    If you’re always in a reactive mode, putting out fires that could have been avoided, the impact to revenue and customer satisfaction is only one half of the story. The revenue not generated, and customers not acquired, from improvements and new features, is the real problem for a business’ long-term health. Ultimately, browser tests give teams more freedom to focus on developing new features rather than manually reviewing functionality or redefining test frameworks over and over again. And in a distributed, work-from-home, always-online world, new product development is what wins the day.