devopsy.kinsta.cloud

Tag: test automation

  • How Mobile Testing Converts to Greater Earnings

    How Mobile Testing Converts to Greater Earnings

    From a business’ sales strategy to the type of coffee it decides to stock in the company lounge, every decision boils down to this: How will it impact this year’s earnings? That’s what makes testing critical for any business with a mobile app: Having a poor-performing app is just like throwing money away.

    There are three primary ways mobile testing translates into more cash for the company: corporate image, performance and speed to market. Read on for the details.

    Testing and Corporate Image

    Treat your mobile application like the face of your brand.

    The majority of time people spend on anything digital is over a mobile application. Users spend more time on their smartphones than they do on their desktop and even television. From a business perspective that means any customer, both real and potential, will experience your brand over a mobile app more than on your TV commercials, physical billboards and even website.

    They will see you over their phone more than in your store.

    devops1

    Mobile testing makes sure that the lights are on in your business, and that the air conditioner is working. We take for granted that when we enter a place of business everything is working properly. If something is off, we immediately recognize it. The same holds for a mobile application. If a function is not working right or is performing slowly, it will impact the user’s decision whether or not to become a customer.

    If it is working perfectly, the larger audience that mobility commands will generate a larger customer interest in your brand.

    Testing and Application Performance

    The faster the app, the happier the customers.

    The instinct for testing applications is to make sure the code itself is working. This is the basis of functional testing. In the age of desktop apps that work directly over the computer’s operating system, or even web apps that work over a local connection, the speed at which your application moved was usually assured.

    Mobility has changed the rules. A mobile app working on a portable device can operate over any of the 400 mobile networks worldwide. It can have to brave poor network conditions, being confined to an elevator, or even riding through a tunnel. All of these different environments affect the speed at which your functions execute, which is no longer static.

    devops2

    If a function moves to slowly, the app will be considered broken. In fact, 90 percent of the mobile applications on a phone right now won’t be there in a year’s time. This is the need for performance, namely network virtualization testing: It makes sure your application provides potential customers with a smooth experience.

    A smooth experience is the best way to guarantee return business, and retaining a customer is 5 times more cost-effective than acquiring a new one.

    Testing and Speed to Market

    Time is more important than money—with the right tools.

    There are 20,000 mobile devices out there, running on four different type of mobile operating systems, all with their own set of versions both new and old. Every day there are devices and operating systems enhancing their capabilities with expanded capacity and new features. This demands that your mobile application adapt by including new features and capacities that capitalize on what the platforms are now offering.

    The key to DevOps is to develop, test, and release the most cutting-edge version of your mobile application as quickly as possible without sacrificing quality in speed or execution.

    devops3

    Even if it is for only a short time, offering features your competitors are not ready to release is like having monopoly on a part of your market: You stand alone. Those customers looking for the best app will have only you to go to.

    Mobile testing is the most arduous step in the DevOps process. A quality test automation tool that includes both performance and functional testing, connecting to a mobile lab enabled by a mobile cloud-testing tool, will accelerate your mobile testing while shortening the release time for your latest app incarnation.

    Conclusion

    In the age of mobile, everything that impacts your mobile application impacts your bottom line. Mobile testing reduces expenses, retains customers and enables your development team to design and release an application that can add solid growth to your company’s coffers.

  • Integrated Testing Without Integrated Migraines

    Integrated Testing Without Integrated Migraines

    I am always amazed at how many practitioners (or, rather, managers of) tout the benefits of test-driven development (TDD). This inherently changes the role of the traditional developer to one of developer and tester. But it would appear that as long as I am able to construct a test for the innovation I wish to create, all is well in the universe, and I can proclaim the full success of automated test integration with my DevOps processes, sleeping quite well at night from then on.

    But in the world before yesterday, testers tend to think differently than developers do. They evolved differently, bearing the scars of applications that tested just fine on the developer’s laptop but somehow died under the myriad circumstances that occurred in the world beyond that original safe harbor. In point of fact, human testers with a variety of automated tools tended to very often “prove” to developers that their sanctified code was not in fact so pious, but actually more “holy” (that is, containing holes) than they thought. Thus, remained the cycle of code/test/code again until the gauntlet to production is navigated successfully.

    There is also a certain reality about testing that developers cannot easily overcome. The only category of testing than can easily accompany a new feature or function is primarily functional testing. “Did the function work?” would appear to be the kind of testing a developer could construct to accompany his/her code. Notwithstanding the fox-in-the-hen-house analogy, there are quite a few other kinds of tests that are equally valid in that code moving forward to production. Most of these tests are not easily constructed by the original developers.

    One might consider performance testing a bit important. In this instance, I am not only concerned with the validity of my new feature, I am also concerned with its performance (timing) relative to how a user would experience it, and inclusive of the conditions of my production environment. For this to be rightly measured, I may need to ensure all integration testing also has has been completed successfully. And assuming all of this completes as expected, I may still want users to test my app—you know, to see if it is actually what they wanted in reality (after decrypting the requirements from the ancient Egyptian notes of the stories they submitted). Beyond this, some functions appear largely in a batch context, or perhaps only when proximity is achieved in a mobile app, so context is equally important to testing.

    The Nuances of Integrated Testing

    But getting back to our original question, let us assume I can test something, and let us assume I have tested it. Where do we go from here? Once the validity of my new feature has been tested, should I decouple the test execution from the build automation and move it to a list of regression tests I only execute periodically? That would seem logical, as this test might need to be run again during post-integration testing (particularly if it manipulates data with upstream dependencies). What is the “value” of my test past that first successful occurrence? If it has value, then I must begin to look at all of my testing in the same way I look at code: It needs to be version-controlled.

    Pairing functional tests to the versions of the code they validate will allow me to roll test automation forward or backwards to match the version of code in the target testing environment. So, for example, if coding build version 1.5 is in my integration environment and it fails, I can roll it back to version 1.4 and concurrently roll back the tests that were valid for the older version as well. That’s better than using tests designed for version 1.5 of my code, as testing new feature validity against old code is sure to generate errors.

    But treating test scripts (or entities) the same way you treat code with version controls is something most organizations have yet to get their heads around. Deploying testing as a part of the post-build deployment automation is equally daunting.

    And, of course, none of this addresses the more fundamental question: Does my test have value at all? Because I can test a feature or function, should I test it? Is this feature or function critical to the user experience? Am I testing it in such a way that the user experience will be evaluated? Testing whether a date field allows only valid data to be entered into it seems important the first time we do it. But once the characteristics of a date field have been so ubiquitously defined by software libraries, advanced programming languages or database structures, perhaps continuing to validate that input is excessive and provides diminishing value. Testing the right thing matters as much as testing at all.

    Then let us assume we have found the Rosetta stone of items to test and we have created an ability to test this feature with scripts passed down from Olympus. If the continued validity or performance characteristics of this feature remain important to the user experience, perhaps we have not found only something to test but also something to monitor. Here is where we begin to think two years into the future. If part of my DevOps continuum is an ability to integrate version-controlled testing as a capability, why not integrate version-controlled monitoring, as well? (Here’s where the Ops guys of DevOps finally get interested.)

    Instead of examining testing from only the perspective of the developer chained to the dictate of TDD, why not examine the life cycle of “evaluation” from a more holistic perspective of coding, integration, performance, users and, ultimately, a monitoring point of view. If my monitoring tools are capable of configuration adjustment, parameter adjustment and ceiling- and basement-level adjustment, then through reading external files or by exploiting API’s shouldn’t we be looking to pair our monitoring profiles with the version of applications we deploy into production discreetly by version? Perhaps a less useful test simply gets retired and a more useful test evolves into a program characteristic that we monitor until the feature itself gets retired.

    Integrating the Ops team into discussions about automated monitoring is sure to pique their attention. If nothing else, the ability to turn off monitoring tools during deployments and turn them on again afterward will cut down on “noise,” and help to focus resources only where they are truly needed when alerts emerge. Understanding the difference between how production is monitored vs. the unique needs of the test environments—with the ability to incorporate both into the automation of DevOps—is equally appealing.

    In any case, the evaluation of an application includes not only the original tests to ensure validity, but also the monitoring that occurs to ensure continued performance as expected. Thinking about where tests begin and where monitoring is more effective or needed can help to keep both disciplines modest and focused only on what is critical, rather than both attempting to take on the weight of the world. The goal of DevOps is not to increase the frequency and duration of headaches, it is to eliminate them.

    To continue the conversation, feel free to contact me.

     

  • Continuous Testing: What exactly is it?

    Continuous Testing: What exactly is it?

    ***This post is based on themes and ideas from the great e-book, “Continuous Testing,” by our friends at Parasoft.

    The aptly named DevOps practice of Continuous Testing synchronizes Testing/QA with Dev and Ops processes optimized to achieve business and development goals.

    A few years ago when Agile was all the rage, lightning-fast Dev sprints left the Ops and QA personnel playing catch-up due to inadequate and sluggish testing practices with minimal coverage across frequent builds.

    The widespread culture of Agile development has accelerated development while software testing still lags behind, forcing organizations to cut corners in QA or slow down Dev processes entirely.

    Instead of speeding up the conveyor belt beyond control or slowing down development to kill efficiencies, Continuous Testing in DevOps encourages a systematic approach toward process improvement.

    Continuous Testing goes beyond automation and encompasses all practices — including tooling and cultural change — that help mitigate risks before progressing to subsequent Software Development Lifecycle stages.

    Continuous testing and Automation

    The approach to Continuous Testing can vary and follow diverse pathways to ensure the best user experience is delivered, free of defects.

    It’s barely viable to repeat all tests every time a new feature is added, so the Continuous Testing strategy fosters a company-wide cultural change to achieve four capabilities: Test early, test faster, test often and automate.

    End-to-end test automation practices are intended to integrate QA into existing fast-paced Dev and Ops processes as a means to create continuity while maintaining faster development cycles.

    It’s important to understand the role of automation in Continuous Testing. Automation alone does not enable continuity in testing, but it helps provide a qualitative assessment of risk and practice actionable tasks to mitigate these risks throughout the SDLC.

    In essence, automated testing constitutes the detection process for software issues and defect prevention, whereas Continuous Testing addresses the wider challenge of improving the effectiveness of these detection sensors.

    What does Continuous Testing Look Like

    This example demonstrates how collaboration, automation and tooling enable end-to-end testing that’s performed faster, more often and better aligned with Agile Dev and Ops practices.

    Continuous Testing starts within development processes as Devs use open source tools such as Selenium to test the functionality of their code.

    Tools such as GitHub or other internal repositories can then be used to store tests and version together with the software code. DevOps teams can use the same tests to integrate frequent builds of the software with the Load Impact API.

    Once the code reaches preproduction, dedicated QA teams can use these tests by modifying dynamic test parameters as appropriate. In the production environment, Ops can reuse these tests for acceptance testing and ongoing post production monitoring.

    Once the application goes live, integration with existing APM tools help keep tabs on server-side app metrics.

    Continuous Testing in this example is broken up into at least four environments: Development environment, Continuous Integration environment, QA Pre-production and Performance testing environment, and the Staging environment. Additional post-live monitoring further enable end-to-end continuous testing.

    Key Elements of Continuous Testing

    Parasoft, a global software solutions company, is one of the great thought leaders in DevOps and Continuous Testing. The company has laid out some key elements of the process in a fantastic e-book, and they’re definitely worth sharing here:

    Risk Assessment: Covers risk mitigation tasks, technical debt, quality assessment and test coverage optimization to ensure the build is ready to progress toward next SDLC stage.

    Policy Analysis: Ensure all processes align with the organization’s evolving business and compliance demands. Primary objectives include:

    • Identifying trends associated with injection of dangerous patterns within the code
    • Augmenting defect prevention practices for high risk areas
    • Isolating risk in target areas

    Requirements Traceability: Ensure true requirements are met and rework is not required. An object assessment to identify which requirements are at risk, working as expected or require further validation.

    Advanced Analysis: Using automation in areas such as static code analysis, change impact analysis and scope assessment/prioritization to prevent defects in the first place and accomplishing more within each iteration.

    Test Optimization: Ensure tests yield accurate outcomes and provide actionable findings. Aspects include Test Data Management, Test Optimization Management and Test Maintenance.

    Service Virtualization: Early, frequent and everywhere testing requires access to real-world testing environments. Service visualization enables access to the virtual form of the required testing stages, cutting the waste time to test environment setup and availability.

    An effective Continuous Testing strategy incorporates all of these elements and forces an organization-wide cultural change to synchronize Dev, Ops and QA/Testing as part of the true DevOps philosophy.

  • The Business Value of Continuous Testing

    The Business Value of Continuous Testing

    ***This post is based on themes and ideas from the great e-book, “Continuous Testing,” by our friends at Parasoft.

    Enterprise software development models have presented many innovative solutions for IT to align development efforts with business goals.

    Adopting new software development lifecycle methodologies is as much a business decision as an IT decision. From an IT perspective, this means the ability to produce profitable products at lower cost without compromising quality.

    Maintaining quality through the code changes to add features and push new releases has been a struggle until the inception of the DevOps-driven Continuous Testing model, which transforms costly, time-consuming IT practices into profit centers.

    The DevOps Patch to Agile Pitfall

    Having already adopted lean Dev and Ops strategies, Agile enterprise IT seems to have unlocked unprecedented efficiency and productivity by delivering better business value at a rapid pace.

    But in all the mayhem and chaos in jumping on the bandwagon of the Agile development, IT teams ended up stockpiling a mass of false assumptions.

    Simply implementing Agile processes and iteratively adding functionalities doesn’t necessarily streamline development.

    With this approach, teams ultimately ignored the true Agile philosophy of adapting, collaborating and planning, and ended up following an accelerated and reinvented version of Waterfall in hopes to develop fast, deliver early and manage issues effectively.

    A change was needed somewhere between design and development, and release and production phases where IT teams were failing to identify software defects and perform fixes fast enough.

    The lack of early, adequate, parallel and continuous testing was transforming IT into a cost center by drawing unnecessary expenses and time investments in reworking software to meet desired quality standards.

    The DevOps approach aims to address this challenge by facilitating continuous testing as an ongoing effort to identify and fix software defects early and deliver the right quality at the right time at the right cost.

    Continuous testing presents the following value propositions to raise business bottom line and open doors to unprecedented revenue opportunities:

    Informed Decision Making

    Business expectations about the reliability and speed of application delivery vary throughout the SDLC.

    Continuous Testing creates a centralized system identifying failures and defects that present business risk to the organization.

    Continuous Testing as a consistent effort provides adequate visibility and useful, real-time data. This can be a huge advantage to businesses, especially when it comes to application performance monitoring and testing.

    The assessment of stability and validity of software builds enables Ops and business decision-makers to trade off decisions between risk and time of release, cost investments, functionality, scope and quality.

    Business Agility and Competitive Differentiation

    Enterprises across all industries have development shops delivering software applications as core components of their service offerings.

    These departments, often driven by DevOps, must focus on churning out new, stable and high-quality software and race to deliver a stream of features that complement primary service offerings.

    For instance, mobile check deposit was a leading driver for bank selection a few years ago and financial institutions offering this online service (while doing it well) early gained valuable competitive differentiation.

    In the DevOps world, Continuous Testing creates a safety net for DevOps teams to develop, test and release new features to the market faster without compromising quality. This safety net works as a trusted testing environment, suite and tool to assess the impact of code changes accurately and quickly.

    With these capabilities, organizations gain the agility to bring new products to market and compete on innovation and not on the scale of business operations and investments.

    All in all, great DevOps levels the playing field between the eight-person startup and the massive enterprise competing in the same space. Who releases faster with a better product is more dependant on the process, not the funding.

    Stronger Business Performance

    IT performance is business performance. It’s measured in terms of the quality, productivity and functionality of IT solutions delivered for internal business operations or as marketable products.

    The primary objective of Continuous Testing is to ensure high quality software products are delivered on time, cost and scope. To achieve these goals, IT teams need capabilities such as automation-driven, self-testing builds in a clone of production environments.

    QA is expected to leverage frameworks and automation tools that allow testing early during the lifecycle. That shrinks time spent on test setup and maintenance, using service virtualization to replicate unavailable testing environments and establishing a comprehensive quality validation process with continuous integration systems.

    These practices work as core elements of Continuous Testing to test faster, sooner, more often and continuously, thereby boosting IT and business performance.

    These business values not only generate unprecedented new revenue streams but also unlock cash trapped within the business IT.

    The resulting increase in revenue encourages organizations to change their game entirely and pursue new DevOps-driven business models in improving service delivery, scaling operations immediately and competing on fronts of business innovation.

  • IBM Webcast Nov 12: Use Discovery to accelerate testing and virtualization

    IBM Webcast Nov 12: Use Discovery to accelerate testing and virtualization

    Many of today’s business applications offer a multitude of services (APIs) across many different technologies. These same applications also reach out into a web of already-existing services in order to retrieve and update information necessary to successfully deliver the end to end business processes. In many cases, these applications are being built by multiple development teams distributed around the world with architectural approaches and middleware changing over time and yet we point test teams at the mere tip of the iceberg and say, “Test it!”. Testers know the benefits service-based testing and service virtualization bring but to successfully use these capabilities they first have to understand not only the application’s architecture but also the way in which it communicates with everything around it. Finding this information can be a forensic-like exercise; even if there is documentation there is only one real truth – the environment itself. Watching and interrogating the real systems allows testers to understand what’s really happening under the surface, sometimes with surprising results.

     

    Date: Wednesday, November 12th, 2014

    Time: 1:00pm – 2:00pm eastern standard time

    Registration link: https://ibm.biz/BdEqEf

    About the Speakers:

    Join Nick Portalski, IBM Development Manager for Integration Testing and Service Virtualization solutions as he explains the rationale for using various discovery techniques and explains how their use can accelerate project initiation making it possible to reduce testing cycles and mitigate risk. Al Wagner will be your host for this event. Al drives thought leadership, strategic initiatives, and tangible solutions with a specific focus on quality management, test automation, and service virtualization.