devopsy.kinsta.cloud

Tag: quality

Does DevOps Improve or Undermine Quality?

In my earlier blog, “From QA to Continuous Testing,” I explained that “several industry developments converged to make continuous testing the winning method for software engineering quality assurance (QA).” The “combination of improved test methods, agile processes, continuous integration speed afforded by automation and orchestration tools and low-cost test dynamically configurable infrastructures afforded by virtualization has resulted in DevOps.”

However, a recent survey by Parasoft indicated that some important concerns such as non-functional requirements (NFRs) (examples are: reliability, testability, availability, resiliency) are being overlooked in many organizations that are operating fast-paced Agile and DevOps environments.

Some results gleaned from Parasoft’s survey run counter to popular expectations:
1. Agile teams are less likely to monitor NFRs than non-agile teams.
2. Ownership of quality, especially for agile teams, tends to be QA team members.
3. Business members think NFRs are being measured but technical folks do not.
4. Confusion and disagreements exist regarding who owns NFR objectives.

The report implies that quick Agile and DevOps processing cycles steal time from broader QA concerns.

The Question

So what is the truth? Do agile, DevOps and continuous testing improve QA, or do they undermine quality?

My own observation is that DevOps can improve or undermine QA. The outcome depends on whether an organization explicitly takes a holistic view of quality that assures all of the “ilities” are taken care of during agile, DevOps and continuous testing.

A complete DevOps environment requires a more sophisticated view of continuous testing than simply “shift left,” “fail early,” “fail often” and “test fast,” which are four popular tenets espoused by the DevOps community. A fifth tenet, “relevance,” must also be fully considered. With “relevance” as a tenet of DevOps QA, the entire DevOps and continuous testing pipeline has woven into it, from end to end, the concept that all of the quality aspects relevant to customers are emphasized in continuous testing and analysis of test results. When relevance is fully considered as an equal tenet to the others, then NFRs and “ilities” are not forgotten and the tools, goals and tests explicitly take into account all aspects relevant to customers’ perception of quality. What is more, when relevance is applied to continuous test selection and results analysis, the system not only produces better quality but it also is more effective and efficient.

At Spirent we think that a holistic approach to DevOps and continuous testing, including “relevance,” is critical for successful DevOps. When continuous testing is implemented with relevance as an equal tenet, quality is improved.

You can read more about our views at Spirent.com/solutions/DevOps

What are your experiences and suggestions?

April 4, 2016
Mixed Martial DevOps vs.The Scrum Master

Royce Grace won the most tournaments in UFC history before he retired. With a record of 14 wins, 2 losses, and 2 draws, it’s understandable why he’s considered by some to be the most influential figure in the history of modern MMA. With 100% of his finishes from submissions, he’s undeniably Brazilian Jujitsu royalty. You’d be hard pressed to find anyone familiar with the fighting world that would argue against building a strong base in BJJ while learning to fight. Royce proved that BJJ was the single most effective form of fighting in the early nineties by submitting opponents much larger than him, like Dan Severn, in UFC 4. Severn had just shy of a 100 pound weight advantage on Gracie but ended up getting caught in a triangle choke. I can remember watching the video and trying to figure out how he made him tap like that. All the sudden fighting wasn’t just about kicking and punching anymore, the game had changed forever.

Fast Forward 12 years to UFC when the Master Royce Gracie is to make his return to the octagon. Matt Hughes crumpled him up like a paper napkin. Royce had continued to practice BJJ since his fight with Dan Severn as it was a successful strategy. The mistake he made was investing in a monoculture. Royce kept a razor point on his BJJ, but fighters like Matt Hughes were smart enough to put an edge on all the dull perspectives of their game. Hughes was a well decorated submission wrestler that knew he needed to work on his striking and maintaining his strength and endurance. Ronda Rousey made it clear after her last victory that she’d been working more on her striking too, which makes sense considering she’s already an Olympic medalist in judo.

At the end of the day it’s a fighter that wins a fight, not the discipline they practice. A well rounded fighter is going to be able to adapt to adversity better than someone that only has nth level experience in a particular area. The same is true for software methodology. A scrum master’s team will consistently be outperformed by a team taking a mixed approach. Not because something is inherently wrong with scrum, but that people get the work done, not the process. A team embracing a diverse approach is more likely to do what works. They will be open to more experimentation to figure out what is successful and to confirm when the current model isn’t working. Sometimes it’s the people not cooperating, but telling them there’s a problem because they’re not fully embracing the methodology is usually wrong. Often enough the methodology is being taken as gospel and is embraced too tightly.

I both like and understand kanban, but feel scrum has too much convention to adhere to specifically. I checked Google and it seems to agree with me:

Just to be fair, I checked to see how Google felt about kanban.

But Google didn’t seem to treat kanban as harshly.

If you can’t take my word for this maybe you’ll listen to someone else?

Just the other day Andrew Clay Shafer posted this on Twitter:

I did ask Andrew to offer an alternative approach if scrum wasn’t going to work, and he directed me to slide 46 of a presentation he did in Pittsburgh for a Ruby User Group (shown below). This slide advocates a mixed approach as well. I agree with everything Andrew encourages there, and can honestly admit I need to dig into XP technical practices a little deeper myself. This will not be an abandonment of current methodology and a complete shift in another direction, but an evaluation to see if I can fill some holes in my game.

July 14, 2014
The Failings of Multitasking

I often find myself coaching colleagues (and reminding myself) about managing down the number of simultaneous tasks or work assignments. Everyone in IT knows that demand always outstrips the capacity of IT to take on work. IT is in demand by all parts of the organization, often at no direct or hard cost to departments or individual staff, which can balloon workloads and exacerbate the growing backlog of requests.

This capacity limit, or constraint, to do work isn’t the only reason IT continues as the big bottleneck in organizations. I believe the “silent killer” is the wasted cycles, or “overhead”, that comes with taking on too much concurrent work. Said another way, one of the greatest challenges of managing work is managing work-in-progress within IT. This is one of the reasons the Kanban approach brings great benefit to IT organizations struggling to stay above water.

The Context Shift Tax

We frequently pride ourselves on our ability to multitask but there is a cost, or tax, we pay when juggling too many work items. Each time we pause on one task and shift to another, time and effort is required to reset, reload and return to a point we can productively make progress on that work. Think of this as a context shift that functions much like paging between memory and swap space in a computer operating system. If you’re multitasking 5 items, each swap to a different task requires another knowledge context shift. Expand the list to 12 or 15 work items, add the interruptions we all experience, and we’re potentially spending more time during the day making those mental shifts than doing actual work.

A legitimate argument can be made that over-multitasking increases complexity and decreases quality. As our mental bandwidth is split across too many work assignments, focus and concentration pays a price. We also experience a form of memory loss — some details are forgotten over time, even what the original problem was or why we are doing the task.

IT organizations function much in the same way — shifting from project to project, task to task, as priorities change, new needs arise and existing work is delayed. As the list of priorities and projects increase, the organization and its people expend more and more on this context shift tax, wasting energies multitasking, losing focus, extending delivery times and decreasing customer satisfaction.

Kanban – Keeping a Lid on the Multitasking Dragon

Kanban’s secret sauce isn’t just about organizing work differently, it’s about thinking about work differently. Kanban provides a visual image of work in any system, how it flows, and where bottlenecks appear. The true secret sauce is in applying a limit or constraint to work in progress. If you’ve tried kanban, you know what I’m talking about. Much has been written about the profound effects of limiting work in progress.

Using Kanban we can decrease multitasking, both by individuals and the organization. This reduces the context shift tax, freeing up cycles to maintain focus, perform work and complete tasks. I like to think of it as keeping a lid on the multitasking dragon.

Limiting work in progress means work items are completed, in shorter time frames, with fewer context shifts. Imagine spending 5% or less during the day making context shifts, versus a heavy multitasking load that could consume 20% or more overhead when shifting between work. Less or no memory loss is experienced, decreasing complexity and reducing the opportunity for quality to suffer. By limiting multitasking we also free up cycles to focus on measurement and improvement, essential ingredients to any truly successful IT organization.

April 8, 2014