Category: Continuous Testing

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

  • Styra Makes Source Code Available for Enterprise Edition of OPA

    Styra Makes Source Code Available for Enterprise Edition of OPA

    Styra is now making the enterprise edition of the Open Policy Agent (OPA) available under a source license that enables IT teams to modify or change the underlying code as they see fit when integrating it with other applications.

    Enterprise OPA is a drop-in replacement for OPA, agent software for implementing policy-as-code that is already available under an open source license. Styra’s enterprise edition provides access to a version that provides better performance, native integrations for a suite of data sources and additional capabilities such as policy decision analysis.

    Previously, Enterprise OPA was only available in either a binary or container format. While these formats enable Enterprise OPA to be widely used, they limited it from being built directly into applications for performance optimization because no access was provided to source code.

    Chris Hendrix, director of product management for Styra, said the source license also provides organizations with an assurance that they will be able to continue to modify Enterprise OPA in much the same way as they modify the open source edition.

    It’s not clear how many DevOps teams are managing policy-as-code within their pipelines, but as application security regulations become more stringent, it’s only a matter of time before most of them do so in one way or another. Styra’s approach relies on a Rego programming language that some DevOps teams have found challenging, however. To make it simpler, the company is working on a low-code approach that would help make Enterprise OPA more accessible, said Hendrix.

    In addition to encouraging more DevOps teams to use Enterprise OPA under a source license, Syra is clearly moving to make it more attractive to replace existing instances of OPA with a more robust implementation licensed from the company. No one knows how many instances of the open source edition of OPA advanced under the auspices of the Cloud Native Computing Foundation (CNCF) are running in production environments. But as those applications become more complex, the need for deeper levels of integration enabled by a faster version of OPA becomes more apparent.

    OPA enables developers to programmatically embed compliance controls without waiting for a compliance team to make them available. That ultimately makes it simpler to ensure that the proper controls have been implemented before an application is deployed in an on-premises IT environment. That capability also makes it easier to document which controls have been applied to make any audit less time-consuming.

    Shifting responsibility for compliance controls to the left also enables DevOps teams to better understand which controls address specific regulatory mandates incorporated into most applications versus which controls might be specific to a specific regulation. Armed with those insights, it then becomes simpler to incorporate policy-as-code within a DevSecOps workflow.

    Hopefully, more applications running in production environments will soon be secure by default. Naturally, one of the first places to start is by making sure the compliance controls needed to achieve a baseline level of security have been pervasively implemented.

  • Driving DevOps Excellence: Implementing SLOs for Your DevOps Team

    Driving DevOps Excellence: Implementing SLOs for Your DevOps Team

    In today’s fast-paced world of software development, when continuous deployment and frequent releases are the norm, DevOps teams are essential to facilitating smooth communication between development and operations. In a blog post on LinkedIn, I talked about the importance of service level objectives (SLOs) for the testing team to improve the overall product quality. On top of that, let’s look at how adding SLOs to your DevOps team can improve productivity, dependability and client happiness.

    The Role of SLOs for DevOps Teams

    • Aligning DevOps Goals with Business Objectives:

    SLOs help DevOps teams coordinate their efforts with business objectives. The DevOps team can concentrate on delivering real business value by setting precise performance indicators, such as deployment success rates or infrastructure provisioning timeframes.

    • Promoting Collaboration and Accountability:

    By implementing SLOs, several stakeholders—including development, operations, quality assurance and business teams—are encouraged to work together. Over each stage of the software development life cycle, this shared responsibility promotes a sense of ownership and accountability.

    • Driving Reliability and Stability:

    SLOs play a key role in ensuring the stability and dependability of the system. You can be sure that your services consistently fulfill client expectations when your DevOps pipeline adheres to defined SLOs.

    • Proactive Issue Mitigation:

    Monitoring and alerting systems are used together with SLOs. The DevOps team could avoid service outages and downtime by aggressively identifying possible problems and resolving them before they become more serious by regularly monitoring important indicators.

    • Data-Driven Decision Making:

    Decision-making is driven by quantitative data. With the help of such indicators, the team is able to identify bottlenecks, prioritize improvements and streamline processes based on quick feedback.

    Now, let’s explore some key areas where DevOps teams can set SLOs to improve their performance:

    1. Continuous Integration (CI):
    SLO: “xx% of builds complete within Y minutes.”
    Measurement: Monitor build times and queue times regularly.
    Action: Optimize CI infrastructure and configurations to meet the SLO.

    2. Continuous Deployment (CD):
    SLO: “xx% of deployments are successful.”
    Measurement: Track deployment success rate.
    Action: Improve the deployment process to meet the SLO and reduce deployment failures.

    3. Infrastructure Management:
    SLO: “xx% of infrastructure is provisioned within Y minutes.”
    Measurement: Monitor infrastructure provisioning time.
    Action: Optimize infrastructure provisioning scripts to meet the SLO.

    4. Monitoring and Logging:
    SLO: “DevOps tools and system uptime should be at least xx%.”
    Measurement: Monitor the availability of the DevOps pipeline, deployment system and other tooling, including monitoring and logging systems.
    Action: Ensure high availability for DevOps tools and components.

    5. Artifact Management:
    SLO: “Artifact retrieval time should be less than x seconds on average.”
    Measurement: Monitor artifact retrieval time and availability.
    Action: Optimize artifact storage and distribution mechanisms.

    6. Testing and Quality Assurance:
    SLO: “Code must have at least xx% unit test coverage.”
    Measurement: Track test coverage regularly.
    Action: Encourage developers to write more tests.

    7. Security and Compliance:
    SLO: “xx% of compliance checks must pass.”
    Measurement: Monitor the results of compliance checks.
    Action: Ensure necessary security measures to meet the compliance SLO.

    8. Standardized Tools Selection:
    SLO: “xx% of teams must use the approved CI/CD tools stack.”
    Measurement: Track the percentage of teams using the approved tools stack.
    Action: Encourage teams to adopt standardized tools and provide necessary training and support.

    9. Training and Skill Development:
    SLO: “xx% of team members should undergo relevant DevOps training annually.”
    Measurement: Monitor training completion rates.
    Action: Provide training opportunities and resources to help team members enhance their skills.

    Initially, teams should baseline which percentage values to track by observing the current state. In the event they don’t have the bandwidth to baseline, start with any reasonable arbitrary number over a period of time, and it will be automatically refined.

    Implementing service-level objectives empowers DevOps teams to focus on delivering reliable and high-performing services that meet user expectations. By setting clear performance and reliability targets, teams can proactively identify and resolve issues, leading to improved collaboration, efficiency and overall user satisfaction.

    SLOs are not rigid constraints but are a means to drive continuous improvement and foster a culture of excellence within the DevOps ecosystem. As organizations strive to keep up with the ever-changing demands of the digital world, embracing SLOs is a crucial step toward achieving DevOps excellence and ensuring a competitive edge in the market.

  • IBM LLM AI: COBOL to Java ASAP ¦ ARM IPO is GO!

    IBM LLM AI: COBOL to Java ASAP ¦ ARM IPO is GO!

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Translating legacy COBOL code to a slightly more modern language, and Arm will go public (again) next month. (more…)

  • VMware Preps Generative AI Assistants to Automate IT Workflows

    VMware Preps Generative AI Assistants to Automate IT Workflows

    At its VMware Explore 2023 conference today, VMware previewed what will become a suite of intelligent assistants that use generative artificial intelligence (AI) to automate the management of a wide range of IT tasks.

    Krish Prasad, senior vice president and general manager for cloud infrastructure at VMware, said VMware will eventually build out assistants for managing individual platforms such as VMware vSphere and the integrated VMware Cloud Foundation (VCF) platform through which the company aggregates those offerings into a single platform.

    VMware is creating those assistants, also known as co-pilots, using multiple large language models (LLMs) that the company uses to train AI models using code written by site reliability engineers (SREs), said Prasad. That approach ensures that the recommendations being made along with the code generated are of the highest quality, he added.

    That approach will enable IT teams to identify configuration drift that might be adversely impacting performance or that could create a cybersecurity risk, for example, noted Prasad. The first of those assistants is being made available in beta for VMware Tanzu, a platform based on Kubernetes. VMware is also building an assistant for its NSX virtual networking software that analyzes security events to generate policies and rules that can be applied to firewalls in real-time as threats are identified.

    It will then be up to each IT team to decide whether to implement those recommendations. Over time, it’s clear that the management of IT infrastructure is going to become increasingly automated as confidence in those recommendations increases, he added. In fact, the ultimate goal is to fulfill the promise of ‘lights-out’ IT management, said Prasad.

    In effect, advances such as these will lead to the democratization of DevOps workflows as it becomes simpler to manage IT infrastructure at scale. IT organizations will be able to invest more resources in building and deploying applications rather than hiring specialists to manage IT infrastructure, noted Prasad.

    In fact, the pace at which applications are built and deployed should dramatically increase in the months ahead, Prasad predicted.

    The rise of generative AI creates a unique opportunity for IT teams to become more relevant than ever to their organizations. It will soon be possible to build and deploy a wide range of distributed applications capable of processing and analyzing data at the point where it is created and consumed.

    While generative artificial intelligence (AI) has captured the popular imagination, it’s becoming apparent that multiple forms of AI will soon be used together to automate DevOps workflows at levels of unprecedented scale.

    Machine learning algorithms, of course, are already used by many organizations to automate everything from analyzing log data to identifying security issues to surfacing patterns in tests made using multiple regression, function and unit testing tools. With the rise of generative AI, it is now possible to automatically generate code and create summaries of reports that multiple stakeholders can more easily understand. Generative AI doesn’t replace the need for previous use cases involving machine learning algorithms. Instead, generative AI makes use of a form of deep learning algorithms to create different types of AI models.

    As a result, DevOps workflows will soon be invoking multiple classes of AI models to automate workflows in ways that should enable IT teams to automate processes at unprecedented levels of scale.

  • Google’s Shiny New AI Dev Environment — the ‘Experimental’ Project IDX

    Google’s Shiny New AI Dev Environment — the ‘Experimental’ Project IDX

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Google’s full-stack, browser based development environment in the cloud. It’s not just a Copilot clone, but aims to help you “get an app from zero to production.” (more…)

  • Tricentis Adds Device Cloud Service to Scale Mobile App Testing

    Tricentis Adds Device Cloud Service to Scale Mobile App Testing

    Tricentis has extended its cloud service for testing applications to include the ability to emulate multiple classes of mobile computing devices.

    Mav Turner, CTO for DevOps at Tricentis, said the Tricentis Device Cloud (TDC) eliminates the need for organizations to acquire multiple physical devices to test mobile applications at scale.

    Capabilities include the tracking of key performance indicators (KPIs) such as blurriness, slow downloads, HTTP redirects and network or connectivity issues that adversely impact the user experience, in addition to performance issues such as load times and excessive consumption of CPU, memory and battery resources.

    The grid-based service also uses machine learning algorithms to apply artificial intelligence (AI) to monitor application performance and detect video and audio quality issues.

    Overall, the goal is to make it simpler for DevOps teams to build and deploy mobile applications faster and make sure they are thoroughly vetted rather than waiting for feedback from end users to discover issues that could have been easily identified before an application is deployed, said Turner.

    The launch of TDC follows the company’s acquisition of Waldo, a provider of a software-as-a-service (SaaS) platform that provides IT teams with a no-code tool for generating mobile application tests. Waldo uses a virtual environment to simulate iOS and Android devices in a way that eliminates the need for physical devices to conduct tests. Instead, DevOps teams record their applications and then use the no-code tools to generate tests directly from within a browser.

    In the meantime, the debate over how far application testing should be shifted left toward developers continues to rage. In theory, the more developers test applications themselves the fewer issues there should be once an application is deployed in a production environment. However, there is still going to be a need for dedicated testing teams to conduct tests across multiple stages of the application development process to ensure quality. In fact, as more routine tests are conducted by developers, it should free up application testers to focus more on edge use cases that might otherwise not be tested before being encountered by end users, noted Turner.

    Testing mobile applications is more critical than ever because the cost of failure—in terms of impact on the business—is now much higher as more organizations depend on software to drive business processes. The challenge is that many of these app developers don’t have a software engineering mindset, so they need testing platforms that automatically create and run tests on their behalf in a way that can be customized for specific use cases. In time, many of those tests will be automatically created by artificial intelligence (AI) models that should reduce the cognitive load required to manage the testing process, added Turner. In effect, application testing will soon be democratized by AI, he noted.

    Over time, the quality of applications should steadily improve as many of the low-level mistakes that developers make are eliminated. That should improve everything from application security to the user experience before mobile applications are initially deployed and then continuously updated.

  • Low-Code/No-Code Testing: Prioritizing AI-Enhanced Test Automation

    Low-Code/No-Code Testing: Prioritizing AI-Enhanced Test Automation

    As organizations strive for faster application delivery and uncompromising software quality, traditional approaches to test automation fall short. With the emergence of low-code/no-code powered by Agile methodology and AI-enhanced tools, a new era of efficient and effective QA automation services has dawned. The expectations of the modern digital landscape require test automation services to undergo a profound transformation.

    Low-code/no-code QA automation services represent a paradigm shift in the software world. Its simplicity, agility and AI-driven capabilities enable organizations to embrace a more efficient and effective approach to testing.

    What Low-Code/No-Code Brings to the Test Automation Landscape

    ● Hassle-free test automation: Low-code/no-code platforms empower testers to automate regression and functional testing with remarkable ease, reducing the reliance on complex scripting and coding. This simplicity accelerates the testing process, enabling quicker iterations and faster time-to-market.
    ● Agile alignment and unparalleled speed: Low-code/no-code test automation seamlessly integrates with Agile methodology and ensures that testing keeps pace with rapid development cycles. This alignment enables organizations to deliver high-quality software within shorter timeframes, gaining a significant competitive edge.
    ● AI-enhanced QA automation tools: By harnessing the power of artificial intelligence, low-code/no-code test automation tools can intelligently analyze vast amounts of test data, identify patterns and make informed decisions. This enhances test coverage, improves efficiency and minimizes false positives, resulting in more reliable and accurate test results.
    ● Cost savings and optimal resource utilization: By significantly reducing the need for extensive scripting and coding expertise, low-code/no-code quality assurance testing reduces maintenance costs and frees up valuable resources. Organizations can allocate their testing budgets more effectively, achieving higher ROI on their quality assurance investments.
    ● Uncompromising Software Quality: Through comprehensive and automated testing, low-code/no-code test automation ensures robust software quality. By minimizing the risk of human errors and identifying defects early in the development life cycle, organizations can deliver superior products that consistently meet or exceed customer expectations. This focus on quality helps build trust, enhance brand reputation and drive customer loyalty.

    Why You Should Prioritize Low-Code/No-Code Test Automation Services

    DevOps teams that recognize the potential of low-code/no-code QA automation services and their ability to drive innovation will be well-positioned to lead their organizations to success.

    Here’s what AI-enhanced low-code/no-code test automation specifically brings to the table for DevOps:

    ● Agility for competitive advantage: Agile methodology combined with low-code/no-code QA automation allows organizations to respond quickly to market changes, customer feedback and emerging opportunities. DevOps teams can ensure that they embrace agility, enabling them to outperform competitors and capture new markets swiftly.
    ● Unleashing the power of AI: AI-enhanced quality assurance testing platforms leverage the capabilities of artificial intelligence to drive intelligent testing. DevOps teams can harness the power of AI to enhance test coverage, identify complex patterns and make data-driven decisions that optimize testing processes and ensure high-quality software.
    ● Resource optimization and cost efficiency: By adopting low-code/no-code test automation practices, teams can optimize resource allocation and testing budgets. The reduced reliance on specialized coding skills allows organizations to leverage the full potential of their testing teams, saving costs and achieving more efficient testing outcomes.
    ● Meeting the demands of software quality: In today’s competitive market, software quality is non-negotiable. DevOps teams must prioritize Agile methodology and AI-enhanced test automation to ensure that their organizations consistently deliver high-quality software that meets or exceeds customer expectations. This commitment to quality helps build trust, enhance brand reputation and drive customer loyalty.

  • 2024—Year of the Linux Desktop? ChromeOS Reflects its Inner Penguin ¦ GNOME Rethink

    2024—Year of the Linux Desktop? ChromeOS Reflects its Inner Penguin ¦ GNOME Rethink

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Can the Linux desktop installed base ever break the mythical 10% barrier? Google has been refactoring ChromeOS, and GNOME is working on new window manager ideas. (more…)

  • The DevOps Pendulum: Agility Vs. Control

    The DevOps Pendulum: Agility Vs. Control

    Managing changes to cloud assets is a universal pain felt by many engineering leaders today, despite all of the advancements in tooling and practices like GitOps. This is because, in reality, it is simply impossible to always have everything completely locked down—we don’t live in a utopia with zero incidents. If your engineering organization prevents any changes from being made through the cloud console or to your infrastructure-as-code (IaC) without complying with strict GitOps practices or change management processes via CI/CD, then it’s likely you have very frustrated developers—developers who can’t troubleshoot or debug in real-time and who have little control in a real-world incident.

    Engineering, like everything else in life, is all about balance.

    The Pendulum Swings

    The pandemic created a whole new mindset and practice for managing distributed and high velocity engineering and operations. Overnight, companies that were not built for remote ways of working had to continue their operations in a global, distributed and asynchronous way that they weren’t completely familiar with. This required a new way to think about software delivery—and it accelerated the DevOps practices that support that delivery. Self-service infrastructure removed barriers for developers to ensure continued performance and velocity.

    At the same time, your cloud cannot be the wild west with everyone creating bespoke infrastructure. It becomes impossible to manage, and misconfigurations can be risky. Guardrails and policy automation have become a hot topic. Today, with tech markets depressed and cloud costs escalating, there seems to be a growing trend toward locking things down again, even at the risk of frustrating developers.

    This begs the question: How can you achieve unencumbered infrastructure for your developers while simultaneously following policies and best practices for compliance, risk and cost? There is a way to find balance.

    Like many aspects of security, we’ve learned that when constraints and barriers are too high, users ultimately find ways to bypass them. This is true for operations, as well. While sometimes it might seem easier to lock everything down than design a better and more balanced way to enable developers to move fast, eventually this approach backfires. This is the exact same evolution the application and cloud-native security industry are undergoing right now.  All the guardrails and controls applied have created too much friction in development processes and developers ultimately bypass them.

    Real-Time Insight is Key

    CloudOps can learn a lot from the disruption the security industry is going through today. In the same way that point-in-time security has been rendered completely useless, non-real-time alerting or eventual alerting to infrastructure drift just won’t cut it when managing an ephemeral cloud. What is really needed is the same kind of real-time and continuous scanning of cloud assets and IaC, similar to what we apply to our systems through monitoring and observability. These solutions became an essential backbone of our business to ensure continuous operations and availability of cloud services.

    As we embrace IaC and the benefits it brings, everything-as-code unlocks greater agility and visibility, enabling you to automatically remediate without locking everything down. As DevOps says, “fail forward and fail fast.” Rather than focusing on never making a mistake, focus on how to fix it immediately. 

    The Solution

    By providing a continuous comparison of actual cloud assets with their desired state through IaC and GitOps, it’s possible to surface configuration drift and policy violations immediately, much like any other kind of breach or major system failure. Failures and incidents are inevitable. It’s unrealistic, and even dangerous, to build systems with an inherent underlying design that prevents you from changing something at the cloud console at 2:00 A.M. during an outage.

    With a locked-down approach, a developer would need to wait for the change management approval during a high-pressure production incident, oftentimes at 3:00 A.M. (because the incident gods of always make them happen in the middle of the night) or on a weekend. We don’t need to look much further than the catastrophic 40-minute and $440 million Knight Capital incident to understand that sometimes time is not on our side and that a delay can have severe consequences.

    Cloud drift has become equally troubling as uptime or any other mission-critical aspect of continuous business operations, and that’s why we need to apply the same principles for monitoring drift in real-time as we do to our CPU and load. This will make it possible to be alerted when your cloud and your IaC are no longer aligned and even flag issues that can be prevented in real-time. You can automate ticket creation and escalation and also triage whether this is something that needs to be fixed in real-time with remediation suggestions provided or open a ticket to fix it later. The choice should be yours, not made by an administrator who has a lot less context about the systems and the ultimate business impact.

    It is Not ‘One and Done’

    Some vendors preach codifying your cloud into IaC and then locking it down so that you’ll never have to do it again. This feels like deja vu; we’ve definitely heard this before. It sounds a lot like the ‘write once, deploy anywhere’ JVM promise that never quite panned out. 

    By going down this path, you’re resigning yourself to never adding new cloud assets or changing your cloud configurations. If there’s one thing we’ve learned about the cloud, it’s that it’s a constantly moving target.  (Read what cloud asset management can learn from the world of finance—another industry in constant flux). If this didn’t work decades ago with the JVM, it certainly won’t work in a dynamic and constantly changing environment like the cloud. That’s a fool’s journey.

    Continuous Cloud Up to Code

    There are major downsides of locking down agility and velocity. This is hard-earned advice, witnessed firsthand from tools like legacy CMDB and startups who have forgotten that cloud is ephemeral—and possibly the reason why we all use it in the first place! So if our cloud is constantly changing and is a continuously moving target, then shouldn’t our tools be built in just the same way to keep up? Solutions built for the rapidly evolving cloud-native era must continuously look for new, unmanaged and changed assets and ensure their cloud infrastructure always follows policies and regulatory standards.  

    Once we are able to find the balance between agility and control, we’ll be able to tap into the benefits of speed and safety (as DORA research shows time and again) that are the benchmark of high-performing teams. Automating cloud asset management can provide governance, policies and control while still enabling the velocity and agility engineering organizations today require. 

    Agility Versus Control

    There’s a minority of people who claim they don’t worry about cloud asset configuration drift because they have everything completely locked down. Zero changes can be done at the cloud console or to IaC without going through GitOps for strict change management processes via CI/CD. I bet they have happy developers—not!

    Life is All About Balance

    Throughout the pandemic, organizations had to make developers happy for fear they would leave. So DevOps went full tilt aiming for self-serve infrastructure for developers to remove barriers to speed and velocity. Now, with tech markets depressed, maybe it’s easier to lock things down even if that risks losing engineers.

    As we embrace IaC and its benefits, you need not live in the stone age of locking everything down. How can you achieve unencumbered infrastructure for your developers while following policies and best practices for compliance, risk and cost? There is a way. 

    The solution? Real-time continuous scanning of your cloud assets and your IaC with continuous comparison can surface configuration drift and policy violations immediately. Emergencies are inevitable. It’s bold/stupid/foolish to think you’ll never need to change something at the cloud console during an emergency or that a developer will wait for change management when production is down at 3:00 A.M. on a weekend. It happens, and you must deal with it. 

  • Considering Scriptless Automation: iOS App Dev in 2023

    Considering Scriptless Automation: iOS App Dev in 2023

    According to Statista, over 31,000 apps were released on Apple’s App Store in March 2023. As impressive as that is, it’s considerably smaller than the number of apps submitted to the Apple store during the same month.

    And it’s probably less than the number of apps removed from the App Store.

    Why?

    According to Apple, most submissions fail because of common issues like bugs, crashes, incompleteness, placeholder data and broken links.

    But getting app approval is just one of many challenges faced by iOS developers today.

    Contemporary iOS App User Expectations

    The truth is, even if you slip by Apple, there’s an even greater critic to be concerned about: Your audience. Users are much less forgiving and have high expectations. If your app isn’t perfect, you’ll lose your chance at meaningful adoption and your users will just go elsewhere.

    For iOS users, perfect is simply the status quo, and users have grown to expect it. In practice, this means that apps must be stable across devices and versions, see regular improvements. and provide a great user experience at all times.

    Seamless experience between devices, versions and iterations
    iOS users expect the same experience across multiple devices. From one iPhone to the next, from version to version and from screen size to screen size. It’s the responsibility of developers to prioritize a consistent experience across devices.

    Fast release cycles and regular updates
    Contemporary app users demand feature-rich applications that go above and beyond the simple functions they expect from an app. Given the statistics above, chances are that the competition is fierce. Achieving success usually requires an efficient and speedy release cycle with more planning, more deployments and more testing.

    Easy and frictionless user experience
    User experience (UX) and functionality are one of the big reasons iOS adoption is so strong. People have come to expect high quality from anything under the Apple banner: Users expect an intuitive and trouble-free user experience on every app with which they interact. Every user’s experience needs to be intuitive, issue free and consistent with other iOS applications.

    Challenges Abound

    Meeting those escalating user expectations comes with a wide variety of challenges. For example:

    Device landscape fragmentation
    With so many device types and form factors in circulation, how do you test across all of them? With dozens of iterations of the iPhone combined with an array of versions, screen sizes and configurations, app developers need a way to test on enough devices to cover the majority of their user bases.

    This leads many developers to turn to simulators and emulators to cover devices they lack access to.

    This poses another problem. Simulators aren’t perfect. A simulator doesn’t replicate the functions of the actual device; it can only mimic certain aspects of an iOS device. And it simply can’t effectively mimic hardware, component functionality or network.

    Using a simulator, you can’t test how an application will respond to factors like low battery, a rising network spike, or limited memory, not to mention GPS and camera functionality. You also can’t monitor usage metrics or run meaningful performance tests.

    Real devices present their own challenge. The vast majority of the time, developers will have to worry about the cost of acquiring, storing, maintaining and updating devices. In addition, they’ll have to centralize and control access to them. In an increasingly remote workforce, it’s not a great strategy.

    Manual tests
    Today, most organizations are over-reliant on manual testing. The result? Running test cases becomes the bottleneck that holds up releases. Sure, manual testing is unavoidable in some cases, but their ad-hoc, inconsistent nature leaves room for error. No matter how detailed the instructions may be, people test differently, and that’s a disaster waiting to happen in today’s complex software world.

    Maintaining complex test scripts
    If you’re automating test cases, you’ve experienced the pain of maintaining code-based test scripts. Every release requires coinciding updates to countless unit tests, integration tests and UI tests. The smallest changes can break test cases, causing false negatives and more deployment delays.

    On top of that, you’ve also got to worry about retaining talent that understands your language-specific testing framework. Given the economic uncertainties of today, that’s certainly not a good situation to be in.

    Constant iOS updates
    Each year sees Apple deliver iOS major releases and a variety of minor releases, patches and bug fixes. Organizations have to test after two release cycles: Their own and Apple’s. Failure to do so can lead to compatibility and App Store compliance issues.

    App store guidelines
    As I mentioned above, Apple maintains strict guidelines for any apps published on the App Store. Excellent testing can mean the difference between releasing an app or waiting another few weeks for the next approval. When time is money, this adds up fast.

    Security at scale
    As requirements change, scope increases and features add up, it’s easy to see how it can become more and more challenging to monitor for potential security flaws. In turn, testing needs to be continually updated to reflect the latest security risks.

    A Shift Toward Scriptless Solutions

    Scriptless automation is part of a growing industry of no-code automation tools, often dubbed the no-code revolution. It’s everything test automation is without the reliance on code-based frameworks.

    Scriptless test automation involves recording user interactions with app elements, asserting expected outcomes and assessing the results. The test is then automated, templatized and extended to test other scenarios.

    With scriptless test automation, you can walk through test cases step by step. Some tools even allow you to test on live devices and choose from real iOS devices that are hosted in the cloud.

    Scriptless testing allows you to:

    ● Easily reuse and templatize test scripts. Scriptless automation makes it easier to reuse test cases, create templates and compose new test cases.

    ● Enable faster time to value. Write test cases quicker, make changes without knowing how to write code and integrate directly with your CI/CD tool. As soon as you make changes to your application, your testing team is right there. Some no-code testing tools can analyze your source code and identify testing gaps so that you can address them right away.

    ● Untangle the web of code-based dependencies. With scriptless iOS test automation, testers don’t have to lift and shift their code every time they make a big change. Instead, the no-code framework provides a visual, easily-navigated UI that tells you simply what’s passing, what’s failing and what needs to be updated.

    ● Improve collaboration between teams. Scriptless iOS test automation improves collaboration between testers and developers by making it easy for all teams to visualize test cases.

    Real Device Testing

    When the pain points of fragmentation meet the shortcomings of simulation, developers are wise to test on real devices. At the end of the day, live device testing is the only way to accurately capture an end user’s experience.

    Yet it’s no secret that managing your own device lab is expensive and unrealistic for modern remote work environments. A real device cloud setup solves this problem: A device cloud hosts devices for you that you can access from your desktop, from anywhere.

    Given the ever-increasing complications of today’s mobile app landscape and the incredibly cut-throat mobile app landscape, it’s worth keeping in mind that you may only get one chance at making the right impression on your audience. Getting testing right is the key.

    Into the Future

    Everyone expects more. As a competitive organization, you have to do more with less: Fewer resources, less time and less room for error. Scriptless automation and real device testing won’t guarantee that your app stands out from the crowd, but it will certainly help.