devopsy.kinsta.cloud

Tag: continuous testing

  • From QA to Continuous Testing

    From QA to Continuous Testing

    My prior blog “DevOps – It’s All About Continuous Testing” posits that QA testers don’t need to fear DevOps because continuous testing QA is a central part of DevOps infrastructures that are implemented according to best practices. That is not to say however that continuous testing is precisely the same as traditional QA testing or that QA test engineers can rest on their laurels in the new world of DevOps and continuous testing.

    Traditional QA testing is a serial process that constitutes a distinct phase that follows the development phase and is conducted by an SQA test team “independent” of the development team. The idea was clearly described by Glenford Myers in his seminal books “Software Reliability – Principles and Practices” and “The Art of Software Testing” published in 1976 and 1979 respectively. According to Myers “4th Axiom [of testing] “It is impossible to test your own program”. Myers felt programmers shouldn’t do any testing, even unit testing. While the application of this axiom to unit testing was not followed by most practitioners in the software industry, the idea of independent software functional and system level testing has been a widely accepted practice for many years. The underlying rationale is that it is “impossible” for the same person (the developer) to objectively test his/her own code due to the inherent bias to want the code to work, while a good QA tester must have the opposite attitude to want to break it in order to uncover bugs in the code. If you follow this logic only “unusual” developers would be able to perform both roles successfully and therefore building a team based on this rare personal quality is essentially not practical.

    Despite the obvious logic of “independent QA test” concept, many very complex software projects without an independent QA test team resulted in extremely high quality products. There are many high quality open source software products that had no independent test organization behind each release version. Meanwhile there are countless examples of software projects that had massive independent test resources that resulted in products that failed miserably. How can this be? Perhaps it is not “who” is doing the test that matters but rather it is the quality and frequency of the testing itself that is important.

    Continuous testing, with its fundamental concept of “fail early and fail often”, is the new QA test method of choice. Sure, testing is still essential and QA test experts that are most skilled in the best practices of test design are valued because they will produce the best quality products. Years of software engineering experiences have demonstrated that developers can learn these skills if they are taught and even “normal” developers can do a great job of testing. Instead of QA testing “after development”, testing is conducted continuously as the software is developed, build-by-build, feature-by-feature. Obviously it is better to find bugs earlier rather than later. So why didn’t we always have continuous testing as part of development processes?

    Several industry developments converged to make continuous testing the winning method for software engineering quality assurance. Numerous “modern” test methods such as agile testing, test driven development, behavior test methods, model based testing, and automated regression testing, to name a few, describe how quality software can be accomplished without the need for an independent QA test team. The advent of Agile development processes and DevOps infrastructures, in which development build-test cycles are accelerated to minutes or hours instead of days or weeks, has accelerated the development and adoption of automation and orchestration tools and methods. There is simply no other way to get a reasonable amount of testing done on each build-test cycle unless the tests themselves and the environment for testing are supported by orchestration tools and reliable regression tests are systematically automated as the product is developed.

    Meanwhile virtualization technologies have enabled a wide variety of product test topologies to be stood up on demand at a speed compatible with the rapid build test cycles and yet not require expensive dedicated equipment configurations. It is this combination of improved test methods, Agile processes, continuous integration speed afforded by automation and orchestration tools, and low-cost test dynamically configurable infrastructures afforded by virtualization that has resulted in DevOps.

    Continuous testing is “the new QA.”

    At Spirent we think testing has a bright future in DevOps. You can read more about our views at spirent.com/solutions/devops

  • IBM Webcast Nov 12: Use Discovery to accelerate testing and virtualization

    IBM Webcast Nov 12: Use Discovery to accelerate testing and virtualization

    Many of today’s business applications offer a multitude of services (APIs) across many different technologies. These same applications also reach out into a web of already-existing services in order to retrieve and update information necessary to successfully deliver the end to end business processes. In many cases, these applications are being built by multiple development teams distributed around the world with architectural approaches and middleware changing over time and yet we point test teams at the mere tip of the iceberg and say, “Test it!”. Testers know the benefits service-based testing and service virtualization bring but to successfully use these capabilities they first have to understand not only the application’s architecture but also the way in which it communicates with everything around it. Finding this information can be a forensic-like exercise; even if there is documentation there is only one real truth – the environment itself. Watching and interrogating the real systems allows testers to understand what’s really happening under the surface, sometimes with surprising results.

     

    Date: Wednesday, November 12th, 2014

    Time: 1:00pm – 2:00pm eastern standard time

    Registration link: https://ibm.biz/BdEqEf

    About the Speakers:

    Join Nick Portalski, IBM Development Manager for Integration Testing and Service Virtualization solutions as he explains the rationale for using various discovery techniques and explains how their use can accelerate project initiation making it possible to reduce testing cycles and mitigate risk. Al Wagner will be your host for this event. Al drives thought leadership, strategic initiatives, and tangible solutions with a specific focus on quality management, test automation, and service virtualization.

     

  • Don’t Fear, Testing Team! DevOps is here!

    Don’t Fear, Testing Team! DevOps is here!

    I love Gene Kim’s analogy about how all unicorns used to be horses; its  a great visual that I use all the time when encouraging IT organizations to change from thinking “why not” to “why not us” when it comes to starting their DevOps transformation.  If culture is really one of the greatest enemies to faster adoption, then you have to start with where the horses live.

    One of the biggest herd of horses in every organization is the team of people dedicated to testing and validating applications.  Even though these teams are not being included in the conversation, they are in fact the most pivotal part in the transformation.

    A Dangerous Love Triangle

    DevOps is touted as the marriage of development and operations; a match made in heaven.  One small problem though:  in most traditional organizations there is a love triangle between operations, development, and testing, as the flow of bits roll from one team to the other until the application is delivered to production.  With the conversation shifting to how can development and operations work better together the testing teams are being alienated.  The question is where does a testing team fit into this conversation, when the focus is all about bringing development and operations closer together?  How can the testing team assert themselves to help facilitate this transformation?

    Horn Growing Engagement

    The Phoenix Project is a great narrative that introduces robust DevOps concepts.  One cannot go far on devopsy.kinsta.cloud without seeing “The Three Ways” and what they mean.  The first “way” is about getting a really good understanding of the delivery chain and how it works as a whole to deliver applications to the end user.  The team that knows the most about this is the testing team as they are the glue that binds operations and development together. If you are going to start growing a horn, you need to start here.

    Engage with the testing team early and often.  This team knows more about the flow of information from development to production than any other team.  They understand the process of moving applications from development environments, testing them, and delivering them to production. They understand the infrastructure and how it should work and what operations team needs to deploy applications.  This knowledge is the perfect horn-growing fertilizer.

    Performance-Driven Feedback

    In my last post I talked about how performance and change management were two pillars to start building on.  As you start to move from simply understanding the system as a whole and increasing the rate of feedback, performance data is critical.  Remember, in science there is no right or wrong answer – only data that says the outcome is desired or not. Performance data can help take the emotion out of determining whether or not processes are going right or wrong after change is made.  Most of the performance engineering folks are embedded in the testing groups since this type of expertise is needed to drive a successful DevOps transition. However, be careful as you start to utilize these resources as they have the potential to become your biggest constraints.  Protect them and get the mechanics of performance testing and engineering out of their head and into the heads of your development and operations teams.

    Continuous Experimentation

    Once you have the testing team on board and helping in this process an interesting event should occur.  The testing teams will start to embed themselves into Development and Operations.  This is when an environment of experimentation becomes possible, just like in a factory, quality is being tested continuously. This leads to a more combined focus on the task at hand while ensuring the quality of the applications and the speed of delivery.

    So really the phrase should be: DevOps, don’t fear! The testing team is here!