devopsy.kinsta.cloud

Tag: QA

The Wonder Pill for Mobile Application Success

Dr. James Brown stunned the British Science Festival by announcing a wonder pill that could reduce your weight, lower the risk of high blood pressure, cancer, diabetes, depression, and anxiety. With everyone in attendance eagerly awaiting what it was, he plainly exclaimed that the “wonder pill” was daily exercise.

The most effective solution was the simple one. The same applies towards developing a successful mobile application.

There is a sure fire way to generate greater sales, leap frog the majority of your competition, promote wider brand recognition, and win loyal customers. The wonder pill: make sure your mobile application works.

“The majority of mobile applications get a one star rating from users due to poor quality. Either the apps don’t work properly or they perform too slowly. The best way to guarantee the financial success of a mobile application is to simply test it properly so the user has an ideal experience.” says Tal Barmeir, CEO of Experitest. If a mobile application has a function that isn’t working, 80% users will simply delete it. In some cases they will tell their social media followers about how lousy the company is. They won’t pan the app mind you – they will put down your entire company. Performance is no different. A slow performing application, even if it works, might as well be broken.

If a daily walk is the simple way to quality health, mobile application testing is the simple way to a quality app. If you can guarantee that your application works properly, your mobile application just leapfrogged the majority of its competition that cannot make that claim.

Is Mobile Testing as Simple as a Walk in the Park?

To guarantee top quality of your app you have to test it over all types of devices, and for each device you have to test over several mobile operating systems. For each permutation of device and operating system you need to make sure your application works fast. But be warned! Just because it works fast over a cutting edge local network in Singapore doesn’t mean it will perform as well in rural Indonesia.

Add in smartwatches, other wearables, and apps managing the internet of things and the matrix of test scenarios you need to cover can seem endless.

The best testing tools make it look easy. You can set up an on premise mobile device lab in your company, and using the cloud attach as many devices as you want to it. Each device can have a different operating system, creating for yourself a large a matrix of devices and operating systems, expanding your test area to meet your needs. Your quality assurance testers can access these devices by remote using automation testing tools from wherever they are in the company. A manager can decide to assign devices to different testing teams working in different time zones to create a 24 hour nonstop testing operation. By including a Network Virtualization tool, testers can simulate any local network condition on earth. They can verify that their application works as smoothly in Nairobi as well as it does in New York City. The best testing tools also constantly update to enable you to test for new technologies like wearables and the IoT.

That’s why mobile testing is the wonder pill of mobile application success. It makes your mobile application works. It makes sure your mobile application performs fast. It makes sure your customers and prospects know that your mobile app is the best there is – and they tell their friends about it.

About the Author/David Fink
David Fink is a product evangelist for Experitest, a leading provider of mobile test automation, manual testing and monitoring tools for mobile applications. David worked on Wall Street during the internet boom of the 1990s and 2000s. Watching the same transformation taking place in mobile, he writes about what we can learn this time around to make this new phase in digital growth usher in smoother to benefit everyone involved. Reach him on Linkedin

July 7, 2015
Testing Will Never Be Automated
“Come to think of it, no one speaks of automated programming or manual programming. There is programming, and there is lots of other stuff done by tools. Once a tool is created to do that stuff, it is never called programming again.”

– James Bach and Michael Bolton, “Testing and Checking Refined

That’s one of those points that tends to silence a room, as there’s really no way to successfully argue against it, but since similar positions of frustration came up time and time again at the 2015 STAREAST conference, it obviously deserves some further discussion.

The issue lies in the surprisingly stark differences between the phrases “automated testing” and “test (or testing) automation.” And if you don’t know the difference:
1. You’re not alone.
2. You’re likely not a software tester.
3. You’re discrediting and undervaluing the testing craft.
The good news is that learning the difference is easy—and testers will forgive you for not knowing it until now! They’re very nice people.

Here’s what today’s brightest and hardest working testers want you to know. “Test automation applies tools to testing.” They do not automate testing, and they certainly won’t ever replace it. There’s simply no such thing as automated testing.

And I’m part of the problem. I wrote a column just last year, titled, “Automated testing’s ROI is right under your nose.” I’m somewhat embarrassed that I told the world something that doesn’t even exist provides an immediate ROI—though at least no one went out and bought a completely fictitious tool off of my suggestion.

Judging by the boisterous cheers as numerous speakers at STAREAST pointed out the absurdity of believing that testing can be automated, this belief is something that I predict testers will likely soon make a concerted effort to erase.

As companies focus more on collaborative, cross-functional, and quality-minded efforts like agile, test-driven development, and DevOps, testers have an incredible opportunity to clear up any misconceptions about their immense importance to their business.

Respected tester, author, and public speaker Dorothy Graham recently pointed out a number of these misconceptions in her session, “Blunders in Test Automation” and the very first one on the list was “thinking that testing tools actually do testing.” Sounding almost exasperated at having to point this out, she chuckled, threw her hands up, and said, “They just run stuff, whatever they’ve been programmed to execute, including bad tests.” She almost got a standing ovation.

When you get that kind of reaction from a room of a thousand software testers from around the world—we’re beyond semantics, and are in a real problem. This is a talented, valuable, and understandably fed-up group of people who are being unfairly cost and productivity-measured against software that provides a completely different service.

So how did it get this bad, when entire conferences are dotted with calls to action for testers to clearly differentiate themselves from tools, especially when the differentiation is so easy to see?

One group you may be surprised I’m largely letting off the hook is the tool manufacturers and their marketers. Sure, there are companies that make some claims that testers may wince at, but it’s really hard to find anyone claiming their tool beats, is smarter than, or should replace any human. There’s no reason for them to make these claims; they’re not true, and they certainly won’t help sell the product.

What about journalists and writers like me who cover the software development and testing industries? After witnessing the frustrations of testers when they feel like they’re being replaced by non-human, non-equivalents—you make a mental note to scratch that phrase from your style guide. I hope other writers do the same.

How about product owners and others closer to the business side of an organization? Are they contributing to an under-appreciation of the human aspect of software testing? That’s…a little tricky.

Scott Barber has spent more than fifteen years as a performance tester, but recently experienced (and left) the role of a product owner. While I’m more than familiar with, and a real fan of Scott’s “the truth hurts” speaking style, when he revealed some truths he recently learned, the room grew incredibly uneasy. Many got their first introduction to Scott when he started his session with:

“It’s not all about testing, you know. Testing is not a product. You are not a product…. Products are the source of profit. And as a product owner, I paid very little attention to testers and testing.”

It may sting, but it’s the truth. Scott went on to point out that “there’s no need for certain job titles to know or care how things are built or tested. Nobody will ever ask them that.” However, by testers working to gain a much better understanding of the product and the risks that each release candidate poses, there is a massive opportunity to make the value of their work blindingly obvious and positively revenue-impactful.

Scott’s pro-tip of testers asking a product owner, “Can I do some work with dev and prod so we can get some early observation going on to catch some of these bugs early and head off some of the damage that could come?” drew both a hushed “Wowwww” and “Yeah, I’m using that” from behind me.

This is where the differentiation between testers and their tools should become clear to everyone. The human levels of empathy, ingenuity, and proactive approach cannot be scripted, automated or baked into any tool, and the best testers care enough about their product and its customers to prove it.

About the Author/Noel Wurst

Noel Wurst, managing editor at Skytap, Noel is responsible for setting and executing Skytap’s content marketing strategy. That includes overseeing the creation and distribution of high-quality, targeted content, and managing the team and external resources associated with those efforts. By leveraging deep online marketing, project management, and editorial experience, Noel plays a crucial role in helping generate and nurture leads with compelling content.

Follow Noel on Twitter, @NoelWurst, along with Skytap, @Skytap.
June 9, 2015
BlazeMeter harnesses open source to deliver continuous testing for DevOps

DevOps is all about efficiency and automation—streamlining development and deployment. In order to deliver quality code quickly a culture of continuity is required—including continuous testing. BlazeMeter provides organizations with a platform for continuous performance testing in a DevOps environment.

BlazeMeter claims to be a more affordable—and in some ways more powerful—performance testing platform than HP LoadRunner. The platform isn’t completely unique. It is harnesses and repackages the features and capabilities of JMeter similar to the way that the very popular code repository GitHub is built on the open source Git project.

Open source projects are a very effective engine for creating tools and applications organizations need. The collaborative, crowd-sourced nature of open source projects leads to a solution that is more likely to appeal to a broad consensus. Where open source tools often fall short is in polish and support. Contributors are primarily focused on features and capabilities that impact the tool or platform itself. Organizations, however, also want the metrics and reporting generally provided by an independent software vendor (ISV) as well as the peace of mind of knowing where to turn for support.

That’s where a company like BlazeMeter shifts the value proposition and makes the open source tool better…for a fee. I’m not suggesting there is anything wrong with that. As I noted above, many organizations crave or demand a solution like BlazeMeter rather than the open source tool it’s built on because they need additional features and support.

I spoke with BlazeMeter Chief Evangelist Ophir Prusak about new features and capabilities introduced recently by BlazeMeter. Last week the company launched a new suite of features aimed specifically at test automation for a DevOps environment. The key features include KPI (Key Performance Indicators) Trends, which automatically display graphics of KPIs for easy tracking and troubleshooting; password-free Organizations and Projects folders, allowing DevOps teams to effectively collaborate and execute tests; a Single Sign-On, facilitating quick third party access and authentication; and an improved On Premise Load Generator with newer Docker-based technology, providing increased reliability and a simpler installation process.

Prusak and I spoke about the continuous continuousness of DevOps. We also talked about the blurred line between continuous testing and continuous monitoring. While there are products and services out there that claim to fall under one heading or the other the reality is that continuous monitoring is essentially a form of continuous testing. In that same vein continuous testing—by the very nature of being continuous—also provides organizations with persistent feedback that would alert admins and developers if an issue were present.

In a DevOps world the traditional role of QA shifts down to the DevOps engineers and developers themselves. BlazeMeter integrates with most existing development lifecycle tools and processes so it can work seamlessly.

BlazeMeter takes the power of JMeter and packages it up with comprehensive metrics and reporting as well as enterprise-grade support. Organizations that are striving to develop and deploy code more efficiently should check out JMeter—and then take a look at what BlazeMeter brings to the table to determine if the investment makes sense.

June 3, 2015
Containers: Secure or Not, Here They Are

Containers are here. And it doesn’t mater whether or not containers are a transitional technology (they are, as our JP Morgenthal covered in Containers are designed for an antiquate application architecture) until all applications are designed for cloud and web-scale, containers will be part of the cloud and virtualization scene. Right now, containers are white hot. And as you’ve seen here in the past month, containerization was an editorial focus for us throughout March.

To give you an idea of how popular containers are right now, consider…

–THIS STORY HAS BEEN MOVED TO CONTAINER JOURNAL–

April 2, 2015
Continuous Testing – Accelerated !
Except for cases where test duration is the point of the test, as indicated in my prior blog “Isn’t Continuous Testing a Longevity Test Anti-pattern?”, faster continuous testing cycles are preferred because failing early and often is a DevOps goal. There are multiple reasons for this, but it should be evident that faster tests reduce the wait time to get results and since there are typically many development team members looking for results the developer time savings multiplier effect is huge for every minute of test time saved on each CI/CT cycle. When aggregated over multiple code branches a few minutes saved per test can be man-years of time saved over a year!

So what can be done to accelerate continuous testing? Frankly a thorough answer to this topic could fill a book. Below are some suggestions in a checklist format intended to be useful for DevOps .
1. Organize the team for speed: When team members and the organization are explicitly selected for CT speed the entire DevOps process will be accelerated.
  • A test architect familiar with the principles of best practices test environments and continuous test design is critical to design and tune continuous testing for speed. The architect establishes the CT strategy and speed goal relative to the coverage goal.
  • A test analysis team reviews test results so no time is wasted on results analysis.
  • Developers and test members are trained on best practices test design and their test scripts are peer-reviewed to ensure they comply with the strategy.
2. Select powerful test resources: Powerful, dedicated, scalable resources will accelerate all test processes and shorten the time to results.
  • Select the fastest processors, low latency storage and I/O to initialize and execute both the test systems and system-under-test entities quickly.
  • Dedicate resources that have longer initialization times so they can be pre-initialized.
  • Virtualize resources that can be scaled horizontally without sacrificing speed.
  • If affordable, arrange enough resources to run ALL tests in parallel.
3. Choose test tools that are designed for fast test execution: Look for the following features when you are evaluating test tools. Not only will this save you a lot of headaches in the long term, it will give you the flexibility to design tests that integrate with other DevOps tools.
  • Restful APIs that do not require wait states between commands and responses
  • Priority queuing of test cases
  • Creation of test topologies that can be pre-loaded
  • Ability to spawn out tests in parallel to run on any number of machines
  • The ability to park a test that needs to wait for an event and return to it after proceeding with subsequent tests
  • Caching features that enable initialization and results data to be processed for one test while the tool proceeds to execute other tests
  • Multiple devices, session types and results data processing utilities implemented as an embedded part of the tool and not requiring slow speed scripting
  • The ability to render commonly used test utility functions in a fast execution language.
4. Organize test schedules and test suites to fail early: The sequence and approach for scheduling of test cases greatly affects the speed of testing. If possible the scheduler should launch all tests in parallel as soon as the build is ready. Otherwise priority schemes are needed.
  • Organize the test scheduler and test suites to avoid any unnecessary tests.
  • Cache initialization and topologies data and resources ahead of the next test execution.
  • Check the test environment before running tests, abort if invalid conditions are detected.
  • Organize tests so the most important ones are run first.
  • Set a failure escape threshold and abort the test if exceeded.
  • Eliminate dependencies between tests so that each one can run standalone or in parallel. Queue the next test while a test is running.
  • Use smart gray box testing methods to limit test selection to the most important tests according to the software change and skip tests that are low risk.
  • Run longer tests in parallel with shorter ones.
5.Design test cases to fail early: The test cases themselves need to be designed with the strategy to get test results as fast as possible.
• Design each test case with a single unique test purpose and make it self-contained so it may be scheduled to run standalone or in parallel with any other test case.
• If a test case has multiple verdicts arrange the highest priority verdicts first and close the test at the first failure.
• Prune or refactors tests that have a low failure history.
• Shorten dead-script timers, and make any wait timers conditional so they can expire as soon as the condition is met.
• Reduce the number of test iterations to fit a time budget.
• Reduce communication wait times by using the fastest protocols. Optimize the most frequently used utilities.
1. Arrange test results reduce to minimize analysis wait times: Arrange the test results to be consumable as soon as the tests are run so the system does not bottleneck.
  • Report test results as the tests are run and accumulate the results across parallel test targets.
  • Establish thresholds for aggregated test results and abort the tests and release test resources to be used by other tests if the thresholds are exceeded for any run.
  • Keep track of test timings and adjust the test schedules according the average and peak run times for each test case.
  • Separate test result report processes to persistent resources so no test process is waiting to finish a report and the test process can proceed to the next test in parallel with test results.
  • Correlate the test results to the changes and use trends analysis to help speed up diagnostics and as an input to test selection such that the most failing test cases are given higher weight in future test runs.
The above is a partial list of suggestions for accelerating continuous testing that have been proven to yield good results for accelerating continuous testing and DevOps.

At Spirent we think testing has a bright future in DevOps. You can read more about our views at Spirent.com/solutions/devops

What do you think of these suggestions and do you have others that should be mentioned?
March 10, 2015
Isn’t Continuous Testing a Longevity Test Anti-pattern?

Accordingly to Jim Coplien: “an anti-pattern is something that looks like a good idea, but which backfires badly when applied. “ In DevOps context, continuous testing is an attractive solution for the narrow context of testing specific software components quickly (in cycles measured in minutes or hours) but may be “bad” if the system that the software is a component of has failure modes (E.g., effects of memory leaks) that will only be observed during long term testing over several days and if the time between releases is greater than that. In other words, the pattern of quick test cycles prescribed by continuous testing appears to be a contrary Anti-pattern to the fundamental concept of long duration “longevity” testing. So what are the well formulated positive patterns that are applicable in its stead?

In my prior blog entitled “From QA to Continuous Testing” I indicated “QA test experts that are most skilled in the best practices of test design are valued because they will produce the best quality products.” QA test experts assert that it is unwise to release a software product that has not been tested for durations comparable to the lifecycle of a release. Only tests that are run for a significantly long time compared to the release cadence are deemed sufficient. Just how long is long enough? This is a subject for further discussion beyond the scope of this short article, but my own observation is that best practices test environments run accelerated level load and impulse load traffic patterns on a build candidate for at least 48 hours or more before a release is determined sufficiently tested for at least limited deployment. Even 48 hours is much longer than typical “continuous testing” cycles. So “what positive continuous longevity testing patterns are applicable in its stead?”

Continuous Longevity Testing Patterns may sound like an oxymoron but in fact longevity testing is compatible within the context of continuous testing principles. Below I describe three Continuous Longevity Testing Patterns that are sometimes used.

Pattern 1: Longevity testing time windows are reserved such as weekends. In this pattern the regular Continuous Integration/Test cycles are interrupted during the weekend and longevity tests are only run on the build that was last created prior to the weekend time window. This has the advantage of not requiring any special test resources other than the resources used for regular CT tests and does not require any sophisticated scheduling to orchestrate the test environment. The disadvantage is that the longevity test results are only available weekly, it is hard to diagnose which build caused a longevity failure and problems with release candidates will delay the release for a week until the next longevity test run validates the fixes.

Pattern 2: Longevity parallel testing is scheduled frequently, perhaps as frequently as every few builds, or perhaps daily. This pattern has the advantages that the regular Continuous Integration/Test cycles are not interrupted any time (they run 24/7), the longevity test results are available for more builds than Pattern 1, which reduces diagnostic complexity and may reduce release delays, but has a disadvantage that separate test resources must be reserved for each longevity test in parallel with the regular CT tests. For complex systems under test with many test targets and lots of builds this can be very expensive so there is a tradeoff between test resource costs, and the cost of delays if not done frequently enough.

Pattern 3: Intelligent longevity testing is scheduled only when certain conditions trigger the longevity tests to be started according to smart rules that correlate software changes to problems with longevity performance. This pattern has the same advantages as Pattern 2 and has the further advantage that the cost of test resources is usually much less than Pattern 2 if the rules for the triggers don’t fire too frequently. The disadvantage is the entire scheme depends on the creation of a rules engine and rules that understand the risk of changes for long duration testing. If the rules are too lenient then this pattern might allow too many longevity failures to escape but if the rules are too tight then this pattern is not any better than pattern 2 and yet more complex to implement.

So which Continuous Longevity Testing Pattern is best? The best answer is the one that best matches organizations’ goals for release cadence, cost of testing, and risk.

So indeed, Continuous Testing is a Longevity Test Anti-pattern but fortunately DevOps infrastructures implemented according to best practices have a variety of complementary Continuous Longevity Testing Patterns to choose from.

At Spirent we think testing has a bright future in DevOps. You can read more about our views at
Spirent.com/solutions/devops

What do you think of these patterns or do you have others that should be mentioned?

March 4, 2015
Functional Testing for IoT

The Internet of Things (IoT) is becoming a bit more than a fun three letter acronym. And while internet connected devices have been around for a while, turning it into a practice is an important evolution. It allows the establishment of best practices, better integration into standard software development, and thus more robust applications, sometimes too robust. And because software drives everything, and software quality is key. How can you ensure quality in all those connected devices?

(more…)

February 24, 2015
From QA to Continuous Testing

My prior blog “DevOps – It’s All About Continuous Testing” posits that QA testers don’t need to fear DevOps because continuous testing QA is a central part of DevOps infrastructures that are implemented according to best practices. That is not to say however that continuous testing is precisely the same as traditional QA testing or that QA test engineers can rest on their laurels in the new world of DevOps and continuous testing.

Traditional QA testing is a serial process that constitutes a distinct phase that follows the development phase and is conducted by an SQA test team “independent” of the development team. The idea was clearly described by Glenford Myers in his seminal books “Software Reliability – Principles and Practices” and “The Art of Software Testing” published in 1976 and 1979 respectively. According to Myers “4th Axiom [of testing] “It is impossible to test your own program”. Myers felt programmers shouldn’t do any testing, even unit testing. While the application of this axiom to unit testing was not followed by most practitioners in the software industry, the idea of independent software functional and system level testing has been a widely accepted practice for many years. The underlying rationale is that it is “impossible” for the same person (the developer) to objectively test his/her own code due to the inherent bias to want the code to work, while a good QA tester must have the opposite attitude to want to break it in order to uncover bugs in the code. If you follow this logic only “unusual” developers would be able to perform both roles successfully and therefore building a team based on this rare personal quality is essentially not practical.

Despite the obvious logic of “independent QA test” concept, many very complex software projects without an independent QA test team resulted in extremely high quality products. There are many high quality open source software products that had no independent test organization behind each release version. Meanwhile there are countless examples of software projects that had massive independent test resources that resulted in products that failed miserably. How can this be? Perhaps it is not “who” is doing the test that matters but rather it is the quality and frequency of the testing itself that is important.

Continuous testing, with its fundamental concept of “fail early and fail often”, is the new QA test method of choice. Sure, testing is still essential and QA test experts that are most skilled in the best practices of test design are valued because they will produce the best quality products. Years of software engineering experiences have demonstrated that developers can learn these skills if they are taught and even “normal” developers can do a great job of testing. Instead of QA testing “after development”, testing is conducted continuously as the software is developed, build-by-build, feature-by-feature. Obviously it is better to find bugs earlier rather than later. So why didn’t we always have continuous testing as part of development processes?

Several industry developments converged to make continuous testing the winning method for software engineering quality assurance. Numerous “modern” test methods such as agile testing, test driven development, behavior test methods, model based testing, and automated regression testing, to name a few, describe how quality software can be accomplished without the need for an independent QA test team. The advent of Agile development processes and DevOps infrastructures, in which development build-test cycles are accelerated to minutes or hours instead of days or weeks, has accelerated the development and adoption of automation and orchestration tools and methods. There is simply no other way to get a reasonable amount of testing done on each build-test cycle unless the tests themselves and the environment for testing are supported by orchestration tools and reliable regression tests are systematically automated as the product is developed.

Meanwhile virtualization technologies have enabled a wide variety of product test topologies to be stood up on demand at a speed compatible with the rapid build test cycles and yet not require expensive dedicated equipment configurations. It is this combination of improved test methods, Agile processes, continuous integration speed afforded by automation and orchestration tools, and low-cost test dynamically configurable infrastructures afforded by virtualization that has resulted in DevOps.

Continuous testing is “the new QA.”

At Spirent we think testing has a bright future in DevOps. You can read more about our views at spirent.com/solutions/devops

February 19, 2015
DevOps and Organizational Debt

I recently published a blog on the Perficient Digital Transformation blog entitled “Is Cloud & DevOps Factored Into Your Digital Transformation?” In that blog I discuss the concept of organizational debt and its impact on your DevOps adoption. Here’s what I had to say in that blog:

If technical debt addresses too little investment in upgrading, poor vendor selection, bad platform selection, etc. then, many businesses have also built up organizational debt. Organizational debt could be over regulation, too many low-value policies, to much middle management attempting to hold onto their area of control, and too much governance. If these are not revisited often to ensure that they are continually adding value for the cost of reducing throughput, then that organizational debt will continue to increase backlog.

Interestingly, there’s a lot written about technical debt, but very little has been written about organizational debt, which can have a significantly deeper impact on your ability to continuously deliver than your technical debt.

In working with clients on their DevOps initiatives, we map the current value stream for a given solution area. What becomes apparent from these value stream maps is that controls and policies have been derived within organizational silos with no attention given to how that will impact IT’s overall ability to be agile and deliver more quickly for the business. Moreover, the owners of these controls and policies seem reticent to remove these bottlenecks even after being shown how it impacts business agility.

Here’s an anecdote from one client when asked why they aren’t using snapshots to speed preparation for QA testing. The suggestion was made in response to the client stating it can take upwards of a week to prepare the QA environment for a single round of testing. Their current approach serializes all testing and believes they need to perform a complete regression test due to the resource contention issues. Hence, they cannot test small changes easily and they cannot test often due to these limitations.

“The main issue is that we don’t control the application or database environment, so we need to prepare a QA release and then have those groups perform their duties in preparation of the QA environment for testing.”

Upon hearing this my mind summoned up past experiences working conferences at major venues, such as the Jacob Javitz Center in NY, where you can’t plug in an extension cord or run an Ethernet cable yourself, but you need to use the Javitz Center’s electrician because of union rules.

Is this what IT has become a set of unofficial unions that hold the business in a death grip? This is clearly a case of organizational debt that needs to be addressed.

Of note, I’m not admonishing the various groups for instituting their controls and policies. I’m sure they have emerged due out of need due to past grievances, failures or as part of IT management practices. Regardless, they are now acting as a bottleneck in implementing proven practices for delivering fewer features more often, which have been proven to increase trust and facilitate better prioritization of work.

The result of this organizational debt is lead times have grown to four months or longer to release functionality that is directly related to key business initiatives by the business for increasing revenue.

In his book, “Kanban,” author David Anderson discusses his recipe for addressing bottlenecks and constraints with existing teams. One of his steps is ‘attack sources of variability to improve predictability’. However, this is his last step because, as he notes, “some types of variability require behavioral changes in order to reduce them. Asking people to change behavior is difficult!” And, so it goes, many businesses never address this key constraint because the hurdles of organizational debt have become so burdensome to overcome. It requires a high degree of trust and political capital along with executive sponsorship. As shown by the anecdote earlier, even if the constraint is directly impeding other business priorities it is not easy to overcome.

I believe DevOps practices can help organizations significantly reduce their technical debt. I have seen and read about organizations that have been able to rapidly reduce their lead times while still refactoring and removing backlog. However, the goal of DevOps is to improve the entire flow, not just development, and a good majority of the release aspects of applications will require organizational debt to be addressed to enable improvement across the entire continuous delivery pipeline.

February 6, 2015
DevOps – It’s all about continuous testing !

I loved reading Steve Wilson’s blog post “Don’t Fear Testing Team. DevOps is Here!” https://devopsy.kinsta.cloud/blogs/dont-fear-testing-team-devops-is-here/ The idea that DevOps somehow reduces the need for testing certainly puts fear in the hearts of QA and test teams who may envision the demise of product quality, reduced test resources and even their very livelihood. I understand the sentiment of the idea about testing going away, but this is misleading.

Testing is important to DevOps. Prior to code integration to trunk, developers pre-flight test their code to make sure it doesn’t blow up the trunk. During the CI cycle and code commits from multiple developers integration testing is used to verify merges. During the continuous testing phase where nightly and weekend regression runs are automated for trunk and release branches it is testing that does the job of finding problems before the software release. During the development process the verdicts trends from testing are a primary measurement of progress. During the release deployment phase more testing verifies the release packages are ready for release. Rest assured test team, you are needed. What DevOps really does is bring testing into the mainstream of development processes, and avoids the problems created by having “big-bang” testing left to the end of the cycle, such as release delays, and quality issues.

Testing maturity is a key differentiator of best practices DevOps infrastructures. My experiences with DevOps infrastructures over the years indicate that testing maturity is a key differentiator of overall DevOps infrastructure maturity. Many organizations can automate their integrations, builds and delivery processes but still have trouble with the subtleness of test orchestration and automation. There is a vital role for testing architects and testing teams to offer their expertise for test design, test automation and test case development with DevOps. Whether the organization is using a test driven development methodology, behavior based test creation, model-based testing or whatever they use, testing is a vital part of the overall DevOps process — not only to ensure verify code changes work and integrate well — but to ensure the changes do not blow up the product. So clearly testing is still an integral part of product development and delivery.

At my company Spirent we think testing has a bright future in DevOps. You can read more about our views at http://www.spirent.com/solutions/devops

February 2, 2015