devopsy.kinsta.cloud

Category: Continuous Testing

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

  • Applause Delivers Testing Integration Tool

    Applause Delivers Testing Integration Tool

    Applause today made a bi-directional integration capability that simplifies requesting application code tests of, dubbed Applause In-Sprint Testing, generally available.

    Kristin Simonini, vice president of product for Applause, said application owners will be able to, for example, use a Jira project management application to create a ticket that requests a specific piece of code be tested either by a member of dedicated testing team or by a developer that has assumed greater responsibility for application testing within the context of a DevOps workflow.
    Alternatively, that request could be sent to a tester that is a member of uTest, an online community of professional testers that Applause oversees.

    The company also published the results of a global survey of 1,800 quality assurance (QA), product, engineering and DevOps professionals that found 44% of development teams that have shifted application testing left are spending a full workday every week on application testing. More than half the respondents (52%) noted that amount of time spent on testing is impacting developer productivity.

    On the plus side, a full 88% of respondents that release code multiple times per month said their organizations now test features as they are developed versus waiting until the end of application development process, the survey also found.

    The major benefits of shifting testing left cited by respondents are reducing the number of bugs released into production environments (57%), reducing the cost of late-stage bug fixes (48%), and reducing the need for hot-fixes (39%).

    The top two barriers to shifting responsibility for testing left toward developers is they are too busy coding (45%) and that development and quality assurance (QA) efforts remain siloed (37%). Nearly half of developers (44%) said they have issues with context switching as they shift between coding and testing tasks.

    Simonini said Applause In-Sprint Testing will make it easier for application owners to decide when they want to make use of a professional tester versus having a developer run a test. The Applause Product Excellence Platform then enables IT organizations to leverage a software-as-a-service (SaaS) platform to record a series of test case scenarios that can be automatically converted into an automation script without anyone having to write code.

    The overall goal is to simplify testing at a time when the number of applications being built has accelerated in part because of the increased focus on digital business transformation that occurred in the wake of the economic downturn brought on by the COVID-19 pandemic. As organizations realize how dependent they have become on those applications to differentiate themselves, the focus on quality assurance naturally intensifies, noted Simonini.

    There’s no doubt more organizations are testing with greater frequency. The efforts of developers that previously tended to address large numbers of bugs as part of a series of application updates after deployment in production are now being frowned upon. Organizations realize their applications only get one chance to make a first impression with end customers. As a result, the level of attention to detail being applied to application testing has, for better or worse, never been greater.

  • AI Reimagines Computation and Improves Software Development

    AI Reimagines Computation and Improves Software Development

    While modern business software solutions unlock organizational efficiencies, agility and the ability to deliver new value to customers, such tools were not available when applications deployed on mainframe systems were being developed.

    Today, many of the Fortune 500 companies, the world’s largest banks and insurance companies and several of the largest retailers in the U.S. continue to rely on legacy programming languages, such as COBOL or Java, to power their applications. However, with new languages coming on the scene and experienced programmers in legacy languages exiting the workforce, organizations find themselves in an increasingly difficult spot.

    Is it a Coding Skills Shortage or a Knowledge Gap?

    As proficient programmers retire or move on, organizations across industries are recognizing they no longer have the specialized domain knowledge of the industry and institution to effectively maintain and support their complex critical systems. To combat what is perceived as a skills shortage, organizations are sinking a significant amount of effort into hiring new developers, teaching them to write in the legacy languages and tasking them with changing enterprise systems.

    Modernizing applications, databases and systems that run well in their current state can be risky and expensive, so building on top of and around existing code can be the better choice. But such workarounds are not always completed in the most appropriate manner. For example, when a developer is cutting and pasting large chunks of code just to change one behavior, the end result is often awkward and unmanageable applications that even the most accomplished developers struggle to deconstruct and understand.

    Understanding the code is not enough. Developers have to know what an application actually does, but if they don’t recognize how altering code in one part of a system impacts another part, the consequences can be devastating. As such, the challenge is not a skills problem — most programming languages are straightforward enough to learn — it’s a knowledge problem that exists across programming languages.

    Goodbye, Years of Language-Specific Experience. Hello, AI.

    The programmers who wrote all those billions of lines of code are still running mission-critical applications, but they are increasingly aging out of the workforce or opting for a new or different career path. This results in legacy systems being less well-understood by the companies that rely on them every day. While many programming languages are meant to be easy to read and understand, unknotting the program logic frequently devolves into a tedious reverse-engineering project in which developers are attempting to unearth the intent of previous developers encoded in the software.

    Rather than relying on aging subject-matter experts to pass along that specialized knowledge to programmers fresh out of college (who are in short supply anyway), organizations might consider leveraging advanced artificial intelligence (AI) tools to automate the process of identifying the specific code that requires attention — no matter how entangled that code is throughout the system.

    While AI can be used to help with the syntactical issues of writing code (much like the Smart Compose feature leverages machine learning to “look ahead” and offer suggestions as you type), the impact of such tools is minimal. AI’s greatest time-saving opportunity lies in automating developers’ steps when conceptualizing and writing code. Whatever the reason — be it poor documentation or spaghetti code — developers spend about 75% of their time in search of the source code that needs to be changed. That source code is likely to be interspersed throughout the code repository, requiring a developer to “think through” what the code does, at nearly every step, to trace out the code detailing the behavior that they seek to modify.

    How AI Overcomes the Inadequacies of Automated Testing

    The goal of many organizations is making changes to systems running older programming languages as safely and efficiently as possible. Some organizations are turning to DevOps to improve agility and accuracy without sacrificing their existing mainframe system’s stability and performance. When it comes to COBOL code, where the problem is more acute than in other languages, automated testing in the CI/CD pipeline is inherently inadequate.

    Once a developer has identified the code representing a behavior that needs to be changed, they’re in a position, in that automated pipeline, to ensure that the corrected behavior never changes. However, because the code describing the behavior in question can be spread throughout the system, a developer will make a change believing they haven’t altered any other behavior, but they may have actually ‘broken the world.’ System-breaking changes can be made to the code -and stay hidden – because testing may not cover scenarios that the developer didn’t consider. Well-constructed automated test suites are data-dependent, but assembling all of the data necessary to get 100% code coverage is, unfortunately, next to impossible.

    If, for example, the application required a security or regulatory change, a developer might think that their correction will be covered by the automated test suite when it’s not. Without copious data in the suite, developers cannot guarantee that a behavior will never be altered. With AI “reimagining” computation as conceptual, developers are able to validate that they haven’t broken the system before they compile or check-in the code — or even run it in debug mode to check it — before running the automated test suite. Once developers are confident their change is isolated, the “conceptual check” of the behavior may be leveraged in the CI/CD pipeline, thereby ensuring the behavior remains intact from that point forward.

    In short, advanced AI makes developers’ lives easier because AI can automatically analyze source code and understand the human intent behind it. Now, developers can collaborate with AI to locate source code in an effortless manner, quickly perform comprehensive impact analysis and make the necessary change(s) without ‘breaking the world.’ The improved productivity and efficiency AI unlocks is critical in maintaining and improving the important legacy and mainframe systems that handle tens of billions of business transactions every single day.

  • Perforce Adds Synchronization Tool for Cross-Functional Teams

    Perforce Adds Synchronization Tool for Cross-Functional Teams

    Perforce Software has added a graphical tool, dubbed Helix Sync, that enables individuals, such as artists, to take advantage of the version control capabilities provided by its software development life cycle (SDLC) platform.

    Gerhard Krüger, cloud architect and product manager at Perforce, said that as DevOps workflows continue to evolve in the gaming and automotive industries, the need to manage artifacts such as drawings, for example, alongside software artifacts is becoming more pronounced among cross-functional teams.

    Helix Sync gives those end users access to a desktop client that enables any individual to upload artifacts into a system by dragging and dropping them into the Perforce Helix Core SDLC platform.

    Previously, end users that lacked coding skills would need to share files with developers via cloud services such as Dropbox. The Perforce approach removes the inherent friction development teams would otherwise encounter, while at the same time making sure the artifacts being shared are more secure, said Krüger. In addition, end users don’t have to be concerned about file size limitations imposed by file sharing services accessed via the cloud, noted Krüger.

    It’s not clear how many organizations are looking to centralize digital asset management within the context of an application development process, but the fewer places that artifacts reside, the more productive IT teams tend to be. At a time when the number of application development projects within many organizations has accelerated, there’s a lot more interest in reducing the number of steps any developer has to monitor and track.

    Diagrams and other types of artwork are not the only kinds of artifacts that are finding their way into software development processes. Data scientists are building massive artificial intelligence (AI) models that need to be incorporated into application development and deployment processes. The challenge organizations face is the individuals that create many of these types of artifacts lack the programming skills required to share them via version control platforms typically employed by developers.

    With the launch of Helix Sync, IT teams should assume that, going forward, Perforce will be expanding its overall content management capabilities, Krüger added.

    It’s not clear to what degree workflows will converge as application development continues to evolve. Organizations are starting to appreciate just how dependent they are on software. As that awareness increases, it’s only a matter of time before workflows start to converge around the software development process that are employed to drive multiple digital business transformation initiatives.

    With more of the creators of those artifacts working remotely, the security of those software supply chains has also become more critical. Cybercriminals are becoming more adept at compromising the credentials end users create to access a cloud file service. Software development teams are more likely to employ more rigorous forms of authentication.

    Arguably, there’s never been more artifacts to track during the history of application development than there are today. The thing that makes managing all those artifacts even more challenging is, just like every other piece of software, they are also being continuously updated in ways that aren’t easy to discern.

  • Low-Code Automated Software Testing Drives DevOps

    Low-Code Automated Software Testing Drives DevOps

    The DevOps world has quietly turned its focus to solving an essential challenge: enabling software teams to innovate quickly while meeting high customer expectations for quality. In other words, to build useful things faster with fewer mistakes. This tension has been growing for years, but 2020 added the challenges of remote work and accelerated digital transformation to the continued disruption brought about by cloud, SaaS and DevOps. In 2021, we need a road map for taking test automation to the next level with quality engineering, a discipline that incorporates best practices and data-driven testing into the end-to-end customer experience and drives organizational growth.

    Rethinking the Quality Process for the DevOps Era

    Quality engineering (QE) has traditionally lacked the tools needed to keep pace with digital transformation. Many QE teams struggle with complex, brittle automation frameworks and slow manual testing which create serious bottlenecks to DevOps adoption. While development accelerates, QE is left with disparate or outdated workflows and toolsets.

    Rising to the challenges and opportunities presented by DevOps and digital transformation will require innovation in our approach to quality. More than ever, we need to shift our focus away from quality assurance as a task that you perform late in the software development life cycle (SDLC) in favor of quality engineering practices throughout the entire life cycle. It starts with broad, effective test coverage – beginning when your developer is working on a local branch and continuing until a high-quality customer experience is confirmed in production across the entire surface area of the application – web, API, mobile, email, PDF or otherwise. It also goes deep; you evaluate all aspects of quality, ensuring that you satisfy requirements related to functional correctness, accessibility, performance, user experience and more.

    Low-Code Test Automation Unlocks a Culture of Quality

    With low-code, genuinely effective testing is at the fingertips of every role, from manual testers to developers, automation engineers, product owners and customer support. Requiring all of these roles to develop specialized programming skills is impractical, so low-code tools serve to democratize quality engineering by enabling everyone to participate in authoring tests, updating tests, and gaining insight from test results.

    Low-code testing brings several advantages to quality engineering, particularly in the transition from manual testing to automated testing. While traditionally this process would create major disruptions, low-code platforms are smoothing the way. In their IT Automation predictions for 2021, Gartner expects that by 2024, improvements in automation capabilities will refocus 30% of IT operations efforts from support to continuous engineering. For manual testing teams, that shift would likely trigger talent changes and a long process of pilot projects, retraining, and gradual rollouts. Low-code is key to managing this shift, upskilling workers and ensuring quality throughout.

    Modern Automation Solutions Embrace Rapid Change

    A key challenge with traditional test automation tools is that they were designed for applications that didn’t change frequently and were intended only to validate code. They relied on static selectors to identify page elements. If the selector (say, an ID) changed or objects (the XPath) moved on your webpage, then your tests failed. As more enterprises accelerated development via DevOps, Agile and CI/CD, the burden of test maintenance resulted in the same bottlenecks as manual testing.

    2020 was a breakout year for low-code test automation. New, easy-to-use solutions surpassed the capabilities of legacy frameworks that have dominated the sector. These solutions incorporate machine learning, data analytics and other advanced technologies to greatly reduce the burden of test maintenance and overcome the test rigidity challenges of legacy frameworks.

    With AI and machine learning, test automation solutions are trained by a tester simply navigating their product. As the tester moves through common customer journeys, the solution builds an understanding of each element and its context so that if an element moves or a property changes, tests containing that element are updated without human intervention. This drastically reduces the amount of maintenance required to ensure your tests are current. When tests do need to be updated, low-code models make it easy for anyone to jump in and make the updates. Looking ahead, auto-healing and low-code editing are crucial to supporting quality engineering in high-velocity teams.

    Testing the Entire Customer Experience

    Traditional pass/fail validation stops short of offering real insight into the user journey, a critical step effective quality engineering. The next generation of test automation platforms expands focus from validating code to validating the entire user experience. This starts with broadening testing to cover non-functional requirements, such as application performance, accessibility, and user experience. For example, identifying trends in load time for key pages and monitoring visual changes are done autonomously and essential to managing the customer experience. From there, you can prioritize quality efforts toward the components of your application that are most heavily used or highest risk. This enables you to make data-driven decisions about test coverage — focusing your testing efforts on high-priority pages like the checkout page for an e-commerce site, or high-risk pages such as those with multiple interactive elements that are linked from other pages, such as signup forms.

    But even a fully data-driven testing strategy that only focuses on the app or website UI is unlikely to cover the full end-to-end user journey, resulting in test coverage gaps and potential pain points for customers. For example, a common user journey may include opening an email, clicking on a link, checking a web site, and downloading a PDF. Whereas the limitations of legacy tools relegates testing to the web portion of that journey, modern solutions allow you to validate the entire user or customer experience (CX): receiving and opening an email, clicking on the link, logging into the web site, downloading the PDF and verifying its content. It’s even possible to confirm values in the backend via API – all without writing a line of code. Developing the tools to seamlessly monitor, repair, and manage the expanded CX is central to our innovation and to our customers’ success.

    Quality Engineering as User-Centric as You

    Quality is no longer an afterthought or just a box to check before deployment; it’s an essential part of the user experience, and therefore an essential part of maintaining a competitive advantage in every industry that uses software to serve customers. Test automation is critical for high-velocity teams racing to innovate faster, shift to DevOps, and build quality products while working remotely. Thankfully, a growing number of quality assurance teams are getting a seat at the table as DevOps is implemented, elevating the profile of quality as a strategy and growing demand for test automation solutions that can keep pace with the demands of DevOps.

    In response to this demand, a new generation of intelligent test automation solutions is enabling teams to rise to the challenge by embracing low-code and machine learning to maintain quality. To build a culture of quality engineering in this environment, your team needs test automation solutions that are as user-centric as you are: solutions that provide coverage across the entire user journey. Solutions that not only verify functional correctness, but provide rich insight into all aspects of quality and enable continuous improvement. Perhaps most importantly, you need solutions that enable everyone on the team to participate in quality. This ensures that the entire SDLC actively participates in building user-friendly products that continue to set the standard for quality at the speed of DevOps.

  • Survey Shows Mounting DevOps Frustration and Costs

    Survey Shows Mounting DevOps Frustration and Costs

    A survey of 400 developers and DevOps team members in the U.S. and Europe found, on average, 15 hours a week are being spent managing tasks such as debugging pipelines and waiting for tests and builds to complete.

    Conducted by the market research firm Vanson Bourne on behalf of Garden, a provider of a platform for automating application development and deployment tasks in Kubernetes environments, the survey finds more than three-quarters of respondents (76%) said the time spent on these and other manual DevOps tasks is generally being wasted.

    Garden CEO Jon Edvald said, all told, 39% of a 40-hour workweek is typically devoted to these tasks. When the number of developers and DevOps specialists that are employed today are factored into that equation along with their salaries, as much as $61 billion is being wasted on these tasks in the U.S. alone, he noted.

    The top sources of DevOps frustration cited by survey respondents are waiting for pipelines to run (76%), waiting for builds and tests (74%) and setting up, maintaining and debugging pipelines/automation (71%). In fact, only 11% of all respondents are completely happy with their development setups and workflows.

    Those hidden costs may even increase as organizations bring on additional DevOps expertise. A full 83% of respondents said their organizations are planning to hire more staff or increase their use of freelancers/outsourcing into DevOps roles in 2021. Three quarters (75%) said DevOps budgets have increased or will increase compared to 2020.

    In addition, the adoption of more complex platforms, such as Kubernetes, may also conspire to increase costs, noted Edvald. Respondents at organizations not yet using Kubernetes spend an average of 14.3 hours per week on low-level DevOps tasks, as compared to 16.5 hours a week for those who are already using Kubernetes. Nearly two-thirds of respondents (62%) said their organizations are already using Kubernetes, while 31% are currently evaluating it, to varying degrees. A full 95% of respondents using Kubernetes have faced challenges after rolling it out, including training and onboarding (37%) or complexity of setup/configuration (35%).

    Of course, not all the time that might be saved is going to be applied to tasks that increase overall productivity; nearly half of respondents (49%) said if they had it, they would spend more time developing new products and services.

    Specifically, 46% said they would improve speed and delivery of existing products and services, while 44% said they would focus on improving security for existing products and services.

    Edvald said the survey makes it clear there’s still plenty of opportunity to automate repetitive DevOps tasks. Organizations are spending too much time on the care and feeding of DevOps platforms that takes time away from building and deploying applications, noted Edvald.

    Less clear is when organizations will reach some type of DevOps breaking point. Garden is betting that the transition to cloud-native applications based on microservices will eventually force the DevOps automation issue. However, in organizations that only know the DevOps processes they have used for years, change may be a long time in coming.

  • Okta Adds Free Developer Edition to AppSec Service

    Okta Adds Free Developer Edition to AppSec Service

    Okta this week lowered the barrier to adoption of DevSecOps best practices by making available an Okta Starter Developer Edition to enable developers to embed Okta authentication, authorization and user management capabilities into applications at no cost for up to 15,000 monthly active users.

    Randall Degges, head of developer advocacy for Okta, said interest in identity management among developers is rising as a method for implementing zero-trust initiatives that better secure application environments. Rather than having to embed and maintain authentication and authorization themselves Okta is making a case for developers to invoke an application programming interface (API) to access a service that provides those capabilities on demand, noted Degges.

    The Okta Starter Developer Edition, announced at an online Oktane21 event, reduces the friction developers encounter as they look to embed identity management capabilities, such as multifactor authentication, into their applications, Degges said. Developers can, for example, access enhanced documentation and sample applications as well as new integrations with Kong, Heroku and Terraform platforms and tools, alongside integrations with multiple continuous integration and continuous delivery (CI/CD) platforms.

    In general, Degges said, the shifting of responsibility left toward developers is happening at a much faster rate because the level of appreciation for security among developers is now a lot higher. Identity management capabilities are simply another API that developers can now invoke as part of a DevOps workflow, noted Degges.

    The Okta Starter Developer Edition comes on the heels of the announcement of the acquisition of Auth0, a provider of authentication and authorization services that can be easily embedded within an application. That $6.5 billion deal, however, is not expected to close until summer.

    In the meantime, Okta is moving forward with the Okta Starter Developer Edition that could eventually create a large pool of applications that will one day consume Okta services for a fee as they scale. The more applications an organization deploys, the more likely it becomes that they will have more than 15,000 active users per month.

    In the meantime, the level of experimentation involving digital business transformation initiatives is likely to increase. Developers will be able to demonstrate to their often skeptical cybersecurity counterparts that their concerns have been addressed before an application is deployed in a production environment.

    It’s not clear what impact access to initially free identity management services might have on the adoption of DevSecOps best practices. In many cases, developers will take advantage of a service like Okta simply to address a requirement. Their appreciation for how that service fits within the context of a larger DevSecOps workflow might not be all that high.

    Nevertheless, the state of application security should improve in the months and years ahead. In fact, some developers may opt to update existing applications that, at present, do not meet zero-trust requirements to take advantage of identity management services that can be invoked via an API. Alternatively, the may also decide to replace authentication capabilities that are challenging for them to maintain and update with an API that calls a service. Regardless of the path forward, the historic divide between developers and cybersecurity teams appears to be finally narrowing.

  • Gitpod Revamps Dev Automation Tool

    Gitpod Revamps Dev Automation Tool

    Gitpod announced today it refreshed the dashboard for its namesake open source tool for automating the deployment of development environments, in addition to making Microsoft VS Code its default tool for editing code.

    In addition, Gitpod has included support for sudo privileges and Docker images using an advanced user namespace to map and limit access to workloads using a built-in Supervisor capability.

    Fresh off raising an additional $13 million in funding, Gitpod CEO Sven Efftinge said now that it’s clear developers will be working from anywhere for the foreseeable future, the need for a tool to automate setting up a developer environment on a local machine outside of an office has become more apparent.

    Gitpod claims there are now more than 350,000 developers using its tool that creates a Kubernetes pod on each developer machine to deploy a complete application development environment, including the integrated development environment (IDE), plugins, compilers, build tools, code generators, databases and application servers. Gitpod makes use of the version control systems provided within, for example, a continuous integration/continuous delivery (CI/CD) platform as the single source of truth for configuring an application development environment.

    The overall goal is to eliminate the time developers waste setting up application development environments so they can write code on their own machines. That approach also makes it simpler for developers to collaborate around the same application development environment regardless of where they happen to be physically located.

    Efftinge said Gitpod provides DevOps teams with a tool to automate setting up development environments across any Git repository that is the equivalent of the Codespaces tool that GitHub recently made available. Rather than having that capability for only one repository, Efftinge noted that most developers work on projects that span multiple types of repositories.

    Automating that process also makes it simpler to align development environments with production environments. One of the reasons why code doesn’t work in a production environment when it is first deployed is because the local development environment has been manually configured in a way that doesn’t accurately reflect the production environment. That issue can become even more problematic when developers are working on multiple projects involving, for example, separate microservices constructed using different repositories.

    In the wake of the COVID-19 pandemic and the acceleration of multiple digital business transformation initiatives there’s naturally a lot more focus on automation and developer productivity. Most developers will be working outside the office for the foreseeable future. Of course, most developers have been writing code outside the office for years. The challenge now is enabling teams of developers to develop applications as efficiently as possible, no matter where they are in the world. In fact, now that development teams have become more distributed, organizations are becoming more comfortable hiring developers wherever they find them.

    Eliminating inefficient manual tasks, such as maintaining development environments, is, of course, going to be a prerequisite for making it possible for those developers to, over time, navigate multiple application development projects without ever having met their colleagues in person.

  • Survey Sees Application Testing Shifting Further Left

    Survey Sees Application Testing Shifting Further Left

    PractiTest, a test management platform provider, published a report today that suggests professional application testers have become more adept at working within DevOps workflows.

    The annual survey of more than 1,000 application testing professionals finds that 80% of all application testing within organizations is being done by test engineers, test leads, testers and automation testers. A full 92% of survey respondents also noted they are working within the context of an agile methodology, while 42% cited DevOps specifically. Half of respondents said they work within the framework of a continuous integration/continuous delivery (CI/CD) platform.

    Significantly, more than half of respondents (59%) said testing is shifting left within their organization, but only a third (33%) said programmers participate in the testing process.

    PractiTest CEO Yaniv Iny said that level of DevOps inclusion suggests that, rather than shifting testing completely left on to the shoulders of application developers, a more nuanced approach to application testing is taking hold as professional testers learn to work within the sprints that make up a DevOps workflow. A full 90% of respondents said testing has become more automated within their quality assurance (QA) processes.

    By now, most application testers have come to terms with automation and, increasingly, AI, said Iny. The assumption now is that large swaths of the testing process will be automated, Iny added. A full 97% of respondents said that functional testing automation was either important or very important for them to succeed, while 96% said test automation patterns, principles and practices are also critical.

    Usage of behavior-driven development (BDD) as a testing model has also increased, with 27% of organizations now employing it to go beyond simply testing functions.

    However, the survey also found that, instead of automation decreasing the size of testing teams, those teams are getting larger. The number of organizations with 16 or more testers increased by 10% to 34% compared to a year ago. That suggests organizations are launching either more complex application development projects or launching multiple projects at the same time as they embrace digital business transformation initiatives, Iny said.

    The survey found nearly 70% of respondents have at least five years of experience. Nearly three-quarters (71%) of respondents reported that their income was not affected by the economic downturn brought on by the COVID-19 pandemic. However, nearly half of respondents (54%) reported that they were concerned about job stability, an increase of more than 30% from a year ago.

    The pandemic did change the way testers acquire new skills. The number of testers reporting that they attended online conferences, meetups and seminars increased to just under half (49.5%), up from 40.5% in 2019. Participation in online communities and forums grew to 44%, while only a third of respondents receiving any kind of formal training.

    Testing professionals with five to 10 years of experience earned an average salary of $89,000 in North America, which compares to $76,000 in Australia/New Zealand, the survey also found.

    Application testing as a distinct discipline doesn’t appear to be fading away anytime soon. Rather, organizations appear to be investing even more in testing to deliver higher-quality applications.

  • Report: The State of DevOps Automation

    Report: The State of DevOps Automation

    In the race to accelerate digital transformation initiatives, organizations are encountering more incidents, more downtime, and longer resolution times. In fact, 90.4% of organizations saw an increase in incidents since the pandemic began, according to a recent Transposit report.

    In a completely digital economy, service downtime takes a higher toll. For ITOps teams, working remotely also requires new strategies to organize a team’s response to issues. Amid rising incidents, 94.1% of organizations have increased their focus on service reliability engineering (SRE) practices.

    Transposit just released their 2021 State of DevOps Automation report, which surveyed over 500 IT, DevOps, and SRE professionals across the U.S. The report sheds light on the state of remote work as it relates to incident response processes, reveals increased demand for SREs and highlights the automation they’re using to narrow the resolution time gap. Below, we’ll explore the report’s key takeaways and discover what sort of DevOps automation is trending.

    Remote Work Slows Incident Response

    Try as they might, videoconferencing and group chat solutions like Zoom and Slack just can’t compete with in-person collaboration. According to the survey, 93.6% of ITOps and software teams reported it takes longer to resolve incidents while working remotely.

    Since the pandemic began, 97% of teams have adopted remote work, and 90.4% have experienced increased service incidents. It doesn’t bode well that incidents are increasing as incident response time lags.

    The report findings frame remote work as a top hurdle to resolving issues — 46.1% reported that not being physically in the same room to collaborate on fixing issues was a top concern. But remote work is only one element; others cite changing processes and a lack of visibility into who’s responsible for maintaining code as another challenge.

    Incident Response Hangups

    To make up for these concerns, organizations are investing in incident response strategies. The report found that 56.4% of companies are adopting communication and collaboration tools, with automation tools at 52.3% and integration tools at 43.9%.

    Though the report showed greater investment in those tools, it also identified a persistent communication gap. When conducting postmortems, it can be challenging to correlate human action to incident responses. More than half (57.4%) of respondents felt it’s hard to piece together human actions and team communication during incidents. This context is essential to make sense of mean-time-to-response (MTTR) metrics.

    Interestingly, the report found 96.4% of respondents believe mining insights from human data, like Slack channels or group emails, could help improve incident responses. It appears that having a clearer way to harvest and query incident-related data throughout the entire response process is a universal concern.

    SRE Goes Mainstream

    As I previously reported, the SRE role continues to evolve and gain favor. The report similarly hypothesizes that, based on the survey results, the SRE title is going mainstream. Out of 295 respondents surveyed, 98% said they increased SRE practices in the last 12 months, and 62.4% said they would be expanding SRE efforts in 2021.

    In 2021, SREs will likely remain a hot commodity in hiring. However, these individuals will face mounting complexity in addressing real-time incidents for complex distributed systems. They will also likely be tasked with automating manual processes, as 51.7% of respondents reported that a lack of automation is the top cause of slow incident resolution.

    Automation Goals and Barriers

    40% of companies surveyed now have at least one dedicated full-time staff member working to create in-house tools or bots to automate incident responses. So, what sort of automation is common in the SRE field? The report shows SREs constructing custom scripts to automate things like:

    • Customer communication
    • Recording MTTR metrics
    • Automating infrastructure
    • Corporate communication
    • Auto-generating runbooks
    • Creating or updating tickets

    But, automating these processes is not all that easy. As a top barrier to automation, DevOps professionals collectively cite a lack of documentation. It seems by replacing tribal knowledge with up-to-date, shared documentation, teams could quickly unlock some gains. So, make documentation accessible, and provide copy-and-paste abilities for sample scripts.

    SRE and Automation to the Rescue

    The study demonstrates how the remote economy has increased both the number of incidents and the mean response times to address them. To fill this gap, organizations increasingly turn to site reliability engineering practices to automate incident responses. By leveling up DevOps with SRE, teams could reduce errors and improve overall availability as development teams progressively deliver new features.

  • Low-Code Makes API Testing More Accessible

    Low-Code Makes API Testing More Accessible

    APIs are rising both in quantity and business value. But with that growth comes an increasing number of vulnerabilities, which could affect cloud service dependencies and internal APIs. Functionality and uptime could also suffer as cloud services evolve their APIs with new versions. Thus, API endpoint testing is critical to ensure a reliable back end. With growing interest across the organization to stabilize these integrations, how do we test APIs in a reproducible way that non-developers can grasp?

    Low-code platforms — tools that enable workflows with visual programming models — are opening doors for citizen developers to construct applications in a visual mode. Similarly, the low-code concept could aid API testing by providing an easier method to create API tests to verify functionality and uptime. This could arm quality assurance, DevOps and site reliability professionals with an easier means to test high-value integrations and their UI representations.

    Amid a growing headless movement, APIs are now powering many end-user experiences. However, during a developer drought, it may be challenging to find talent with integration know-how.

    I recently met with Dan Belcher from Mabl to discuss combining low-code and API testing. Here, we’ll uncover why it’s difficult for non-engineers to conduct API testing and consider what API testing must address to ensure today’s software meets tomorrow’s digital innovation demands.

    A Low-Code Approach to API Testing

    The software landscape has evolved rapidly in recent years. “Many forces have combined to create an incredible amount of change,” said Belcher, citing DevOps, CI/CD, the API economy and agile enterprises as examples. In this era, applications are becoming more dynamic, disrupting the way teams conduct end-to-end quality testing.

    API-based cloud services are part of this new frontier, becoming popular building blocks within modern applications. Yet, integrations require upkeep as cloud services evolve and version. Without continuous testing of API-based cloud services, organizations could easily suffer from breaking change and poor usability.

    Unfortunately, as it stands, typical QA teams aren’t well-equipped to handle ongoing regression testing for APIs. Instead, “developers, historically, have been the only ones that understood API testing,” explained Belcher. For years, API testing has been relegated to the realm of developers, who use tools like Postman to perform unit-level testing. But why is API testing so difficult for others?

    What Makes API Testing Difficult?

    Well, to start, non-engineers are typically more accustomed to UI-based testing. These environments use a structure designed for anyone to consume, making it easy to locate fields and confirm values, Belcher explained. When it comes to API testing, you often must handle massive chunks of JSON responses to match fields and surface issues. Testing API responses typically requires writing code, parsing JSON and writing it to variables, Belcher explained, which may be difficult for non-developers to implement.

    Or, difficulties could arise when testing asynchronous endpoints, Belcher added. Asynchronous APIs often involve background server processes that take a while to respond. This comes at a moment when event-driven architecture (EDA) is a growing trend. Take AsyncAPI, a standard API specification for EDA APIs, which recently joined the Linux Foundation. A recent report found AsyncAPI production usage tripled from 5% in 2019 to 19% in 2020. According to Belcher, testing such asynchronous services requires an infrastructure for continuous testing to anticipate these emerging scenarios.

    Four Crucial Areas For API Testing

    So what should API tests be focusing on? Belcher highlighted four areas that warrant close examination throughout the API testing process:

    • Functionality: This is the core functionality. Does the API do what it promises? Does it return the correct data?
    • Uptime: What is the API’s availability? Are all endpoints available in production?
    • Speed: This ensures cloud dependencies meet SLAs. What is the latency? Is there drift in API performance over time? Though API monitors like Api.expert help aggregate industry performances, they might not always reflect your specific customer-level realities. Belcher recommends not placing faith in supposed performance and instead, clocking your own.
    • End-to-End: Belcher stresses the role APIs now play across customer-facing presences. An end-to-end workflow should involve API endpoints, how they interact with one another, and the resulting visual representations. “Just like we have end-to-end testing for UIs, we should have end-to-end testing for APIs,” he said.

    API testing isn’t always combined with other parts of the user experience, but Belcher believes, going forward, end-to-end testing will be critical to ensure all workflows function in concert. For example, if a user profile is created within a browser, teams should verify the user profile has been created at an API level, as well.

    Low-Code API Testing and Shift Left

    APIs are beneficial to avoid reinventing the wheel. However, 100% operational uptime is impossible. Applications will inevitably encounter API problems, including outages, technical failures and functional misrepresentation. Developers also often come to rely upon production behaviors that may not be officially documented, an effect known as Hyrum’s Law. This can cause unstable integrations as APIs version over time.

    Thus, for these reasons and more, it’s a good practice to adopt continuous API testing. With the surging interest in site reliability engineering, spotting API errors early on will become an important aspect in reducing mean time to recovery (MTTR). Testing third-party APIs could also expose production latencies to inform cloud investments.

    “As the pace of change is going up, how do we deliver great quality to our customers?” Belcher asked. API testing is one answer to address this rapid change, and is growing in importance for the software delivery life cycle. It especially rings true for headless frontend experiences, where the API-driven backend isn’t transparent to QA testers.

    Low-code capabilities for API testing could help tear down silos inherent in the development process, enabling more people to participate in testing integrations. Belcher also stresses the importance of improving the usability of API testing for developers and non-developers alike. “We need to integrate with tools developers are already using for verification,” he added.