devopsy.kinsta.cloud

Tag: flow

DevOps’ Three Ways Were Never About Tooling

Somewhere along the way, DevOps became a tooling conversation. Ask someone how mature their DevOps practice is and the answer will often involve CI/CD pipelines, automated testing, infrastructure as code, observability, deployment frequency, or some combination of the above. All valuable tools, but that is often where the conversation ends. They compartmentalize DevOps principles to a niche scenario and ignore those same principles when it comes to team dynamics, organizational design, strategy, product development, and so on.

This becomes more obvious when we return to the Three Ways, commonly summarized as Flow, Feedback, and Continual Learning. These ideas are often explained through software delivery because that is where DevOps emerged, but nothing about them inherently limits their application to software tooling. They describe how information moves through a system, how the system learns what happened, and how that learning changes what happens next.

It is essentially PDSA in action. That should be no surprise, since DevOps gets much of its guidance from Lean, and the same underlying pattern emerges throughout Agile, Product, Design Thinking, and most modern approaches to improvement: plan, do, study, act. The branding may change, but the underlying mechanics are often remarkably similar.

Don’t Stop The Flow

Flow is usually interpreted as the movement of code from development into production. We optimize handoffs, reduce batch sizes, automate deployments, and remove constraints that slow software from reaching users. All of that is useful, but the flow of value begins long before someone commits code. To borrow from John Willis, what does the movement of value look like from aha to ka-ching? An organization can deploy code 20 times per day while taking 6 months to decide what it wants to build. That organization has optimized one narrow segment of flow.

This is a familiar problem in systems thinking. Local optimization can make one portion of a system remarkably efficient without improving the system as a whole. Mik Kersten tells the story of Nationwide in Project to Product. Nationwide measured roughly 120 days of end-to-end flow and found that only 2.5 percent of that time was spent in development. Adding more developers, whether human or AI agents, would have done little to improve throughput because development was not the constraint. Eliyahu Goldratt warned of the same problem in The Goal: “An hour lost at a bottleneck is an hour out of the entire system. An hour saved at a non-bottleneck is worthless.”

Amplify the Feedback

The same narrowing happens with Feedback. Automated tests, telemetry, and monitoring alerts are all common forms of feedback. These loops are valuable because they shorten the distance between an action and evidence about its consequences. The idea comes from Toyota’s andon cord: how do we make an abnormal condition visible quickly enough to stop and respond before producing more of an undesirable thing?

Now what happens when we apply that to value creation? How do we know to stop making something that is not creating the desired value? Value can only be judged by the customer, which means it cannot possibly be known without customer feedback. What is your customer’s andon cord?

As Kent Beck wrote in Extreme Programming Explained, “The development of a piece of software changes its own requirements. As soon as the customers see the first release, they learn what they want in the second release… or what they really wanted in the first. And it’s valuable learning, because it couldn’t have possibly taken place based on speculation. It is learning that can only come from experience. But customers can’t get there alone. They need people who can program, not as guides, but as companions.”

Many organizations become so enthralled with the craft that they forget the reason. Pendo found that 80 percent of software features are rarely or never used, with similar findings published by The Standish Group, Microsoft, and Google. It’s like when Homer Simpson built the perfect car, The Homer, which was loaded with features from triple horns to extra-sized cup holders, only to have the company go under because no one bought it.

Learning Without Boundaries

The Third Way, Continual Learning, is about what we do with that feedback. Every roadmap and business case contains assumptions. Every reorganization, process change, feature request, and strategic plan is built on some prediction about what will happen next. Whether you validate them or not, they are hypotheses until tested.

Learning is not the presence of feedback. It is a change in understanding or behavior resulting from processing that feedback. More flow creates more opportunities for feedback. More feedback creates more opportunities to learn. What we learn can then help us improve flow again. The Three Ways form a reinforcing learning cycle.

The toolchain is where many of us learned the Three Ways, but it doesn’t have to be where we leave them. They apply to customer research, strategic direction, or a frontline worker surfacing a recurring problem. Frequent retrospectives can create feedback about how a team works, while experiments can test whether changing that system produces the expected result. Leadership can apply the same cycle to organizational effectiveness. The principles scale from a line of code to the organization itself. This is the way.

September 11, 2026
The New Convergence: How Value Stream Mapping is Rewiring Product, Platform and DevOps for 2026 and Beyond
For years, organizations have tried to improve speed, quality, and customer experience by layering frameworks on top of frameworks—Agile for iteration, DevOps for automation, ITIL for service stability, TOGAF for architecture, Lean for waste reduction. Each helped, but none solved the core problem: Work still doesn’t flow.

In 2026, the most competitive organizations are converging on a different approach—one that doesn’t start with frameworks at all. It starts with value streams.

Value Stream Mapping (VSM) has re-emerged as the connective tissue that unifies product management, engineering, operations, and architecture into a single, end‑to‑end system of flow. And the results are no longer theoretical. Across industries, and while only as a generalization, many tools, such as DORA et al, teams using VSM as their operating backbone are seeing results such as:
- 25–30% faster time-to-market
- 20–40% reductions in waste and rework
- 2–4x increases in deployment frequency
- Material reductions in technical debt and operational drag
These aren’t magic numbers; they’re the predictable outcome of making work visible, measurable, and accountable across the entire product lifecycle. We know this.

The Modern Lean Product Environment: Where Flow Becomes Strategy

The shift underway is bigger than Agile or DevOps alone. Organizations are moving from project-centric planning to continuous product flow, where teams optimize for:
- Cross-functional collaboration instead of siloed handoffs
- Rapid iteration instead of heavy upfront planning
- Customer-centric learning instead of assumption-driven roadmaps
- Continuous improvement instead of episodic transformation
Lean principles are no longer a manufacturing artifact—they’re the backbone of modern digital operations. And VSM is the mechanism that makes Lean real in complex, multi-team environments.

Why VSM Is the Missing Link Between Agile, DevOps, TOGAF and ITIL 5

Every framework solves a piece of the puzzle:
- Agile accelerates iteration
- DevOps automates delivery
- TOGAF defines architectural guardrails
- ITIL 5 governs service reliability and digital product and service management practices
- Lean eliminates waste
See also;
👉 ITIL Foundation (Version 5) from PeopleCert for the digital product and service management language,
👉TOGAF Standard, 10th Edition – Business Architecture for value streams as a core Business Architecture element, and
👉DORA’s Value stream mapping for software delivery guide for the software-delivery flow.

But none of them describe how value actually flows across the organization.

VSM does.

By mapping the end-to-end flow—from customer need to deployed value—organizations finally see the real bottlenecks: Decision latency, cross-team wait states, redundant approvals, manual testing, unclear ownership, and invisible technical debt.

When teams harmonize their frameworks around a shared value stream, service delivery efficiency can improve materially, and cross-team alignment often improves as well.

MVP + VSM: The Shortest Path to Customer Value

The fastest organizations aren’t just shipping faster — they’re learning faster.

MVP thinking, when paired with VSM, creates a disciplined mechanism for identifying the shortest path to first value. Teams that focus on delivering only the essential, value-critical functionality see:
- 30–50% reductions in time-to-first-value, based on field observations and practitioner reports
- Faster validation of assumptions
- Less investment in features users don’t adopt
VSM makes MVP execution concrete by visually mapping the pipeline, exposing delays, and clarifying what truly matters for early learning.

Feedback Loops as a First-Class Architecture Concern

Persona-driven feedback loops are no longer a UX nicety — they’re a product architecture requirement, for example, telemetry → hypothesis → release decision → adoption signal.

Organizations that systematically connect persona pain points to telemetry, in-app feedback, and post-release signals see:
- ~20% increases in feature adoption, especially in semantic analysis-driven ChatOps and channel management contexts
- Higher retention due to engagement
- Faster iteration cycles due to transparency and shared visibility into common goals
VSM highlights where feedback is missing, delayed, or ignored—and forces teams to close the loop. I have personally seen dozens of times, literally, with relatively simple mitigation and remediation, upon awareness.

Future-State Mapping: The Fastest Path to Deployment Excellence

Current-state maps expose the pain. Future-state maps define the fix.

Teams that redesign their deployment pipelines using VSM routinely achieve:
- 40–60% reductions in deployment lead time
- 2–4x increases in release frequency
- Dramatic reductions in manual work and cross-team handoffs
These gains aren’t theoretical—they’re the natural result of eliminating wait states, automating test-to-deploy steps, and shifting from batch releases to continuous flow. This is what DORA is all about and has covered this area quite well! See DORA’s 2024 report and DORA’s software delivery metrics guidance for deployment frequency and lead time for the latest statistics and information.

Technical Levers: Canary Releases, Feature Flags, Digital Twins

Modern value streams are increasingly defined by technical patterns that reduce risk while increasing speed:
- Canary releases reduce rollback rates by up to 70%
- Feature flags decouple deployment from release, enabling instant rollback
- Digital twins simulate operational impact before changes hit production
Together, these techniques create a safer, faster, more resilient flow of value.

Data-Driven Value Rationalization: Killing Zombie Features

The future of product development is ruthlessly data-driven.

Organizations that adopt hypothesis-driven development, usage analytics, and value realization dashboards consistently:
- Improve decision quality by ~25%
- Reduce investment in low-value features
- Eliminate “zombie features” that drain capacity
- Cut technical debt by up to 40%
This is where product, engineering, and finance finally converge: Value becomes measurable, visible, and accountable. We cover this extensively over at Flowtopia.io and we also welcome direct inquiries to discuss further.

The Real Transformation: From Analog Thinking to Digital Flow

Most organizations still manage digital products with analog mental models—projects, phases, committees, and static plans. But digital work behaves more like microservices: Dynamic, instance-specific, and constantly evolving.

VSM is the operating system that makes this manageable.

It gives leaders a deterministic way to:
- See the system
- Measure the system
- Improve the system
- Align teams around value
- Reduce waste and friction
- Accelerate learning and delivery
In a world where speed, adaptability, and customer experience define competitiveness, VSM is no longer optional. It’s the backbone of modern product and platform operations.
June 22, 2026
You Cannot Fake Flow: What Organizations Get Wrong About Value Delivery

Most organizations that have invested heavily in Agile and DevOps share a puzzling experience. Deployment frequency is up. Teams are busy. Dashboards are green. And yet value still queues. Strategy still takes months to reach the customer. Feedback still arrives too late to change anything important.

Flow coach Marnus Marx has a name for this condition and a useful analogy. “Speed is not flow,” he says. “Responsiveness is not flow. And automation without systemic alignment simply accelerates chaos.” His analogy is butterfly swimming—the most technically unforgiving stroke in the pool. You cannot muscle through bad timing. If arms, legs, and breath fall out of sync, the swimmer stalls, and fighting harder only creates more drag.

Organizations, he argues, work the same way. Agile without flow becomes theatre. DevOps without systemic alignment becomes an expensive tooling investment that speeds up the wrong things.

The uncomfortable conclusion is that most transformation programs have optimized activity rather than outcomes. Teams are faster. The underlying system is not.

Making the Invisible Visible

The discipline that most consistently exposes this gap is Value Stream Mapping. Originally a manufacturing technique, Value Stream Mapping has moved decisively into software and product delivery—but its practitioners are increasingly arguing that its real scope is broader still.

Jack Maher sees Value Stream Mapping as the connective tissue across the proliferating landscape of frameworks that most enterprises now run simultaneously: Agile, DevOps, TOGAF, ITIL, and whatever comes next. The problem these frameworks share isn’t that they’re wrong—it’s that they each illuminate part of the system while leaving other parts in shadow. Value Stream Mapping forces the whole flow into view.

The practical results Jack cites are significant. Future-state redesigns based on Value Stream Mapping have halved deployment lead times in documented cases. MVP strategies informed by value stream analysis have shortened time-to-first-value by 30 to 50%.

Feature flag and canary release approaches, when aligned to a clear map of the delivery flow, have reduced production incidents by 30 to 60 percent. Technical debt—often invisible until it consumes a third or more of engineering capacity—becomes measurable and therefore manageable. Zombie features, the ones nobody uses but everyone maintains, can finally be identified and retired.

Marcin Karkocha extends this argument further, and in a direction that many Value Stream Mapping practitioners haven’t yet followed. Value Stream Mapping’s core principles — making work visible, identifying waste, measuring flow — are not specific to manufacturing or software delivery. They apply wherever value moves through a system. Marcin has used the same technique for IT strategy assessment, pre-sales pipeline optimization, digital sovereignty evaluation, and license portfolio consolidation. “Think of it as the new SWOT,” he says—a universally applicable diagnostic that adapts to whatever business question needs answering.

He adds a practical note on tooling that’s easy to overlook: The right mapping tool depends on the nature of the value stream. A numbers-heavy financial flow often works better in a spreadsheet than on a visual board. A complex, multi-team IT delivery process needs the spatial thinking that a tool like Miro provides. The map itself should be an accelerator, not a bottleneck.

The Practice of Mapping

Understanding Value Stream Mapping conceptually and being able to use it under real conditions are different things. Miguel Dias compresses the gap between them. His approach takes the essentials of a full Value Stream Mapping engagement—exercises that would normally take 10 or more hours— and makes them actionable in a single, focused session: map an actual delivery workflow, measure handovers, identify constraints, and leave with a prioritized action plan.

The key move, he argues, is shifting from gut feel to quantitative insight. Most people have a rough sense of where work slows down in their process. Mapping makes it precise, which changes the conversation from opinion to evidence.

When Flow Isn’t Enough

Here is where the picture gets more complicated, and where Phil Clark enters with what may be the most important—and most frequently missed—part of the argument. Flow improvements, when they happen, tend to happen faster than the leadership systems around them change. Teams adopt better practices. And then funding resets, priorities churn, teams are reorganized, and the metrics revert to measuring output rather than business impact.

“Flow and Realization are both essential,” Phil says. Flow describes how work moves through the system. Realization describes whether the work produced the intended result. Most organizations measure the first and neglect the second—partly because outcomes often have a long tail, and closing the loop requires a discipline that doesn’t fit neatly into quarterly cycles.

His broader point is structural: Leadership choices determine whether better ways of working survive. What leaders fund, protect, measure, and reinforce either sustains the gains or quietly erodes them. AI, he notes, amplifies this dynamic rather than resolving it. An AI-augmented system that is well-led gets better faster. One that isn’t gets worse faster.

It doesn’t have to be this way. The tools exist. The frameworks exist. The measurement techniques exist. What most organizations are missing is the systemic view that connects them—a clear picture of how value actually moves from idea to customer outcome, where it stalls, and what leadership choices either protect or undermine the improvements teams work hard to make. Flow is not speed. Flow is smoothness. And smoothness, it turns out, requires looking at the whole system.

These ideas will be explored in depth at Flowtopia Live on June 24.

Exclusive Offer: As part of the TechStrong community, get a free LIVE pass.

Use Code: TECHSTRONGFLOW

Get your Live Pass to Flowtopia Live 2026 here.

June 19, 2026
From Pilots to Performance: Why Enterprise Flow is the New Competitive Advantage

Digital transformation has never been short of ambition. Enterprises have invested billions in AI initiatives, cloud migrations, agile programs, and customer success platforms—yet many still find themselves stuck: Pilots that never scale, software that sits unused, teams that move fast in isolation but grind to a halt when they collide. The question isn’t whether organizations want to transform. It’s whether they know how to make value actually flow.

That question sits at the heart of conversations happening across the world’s most complex organizations. From aerospace and defense to aviation, industrial software to telecom infrastructure, leaders are learning the same hard lesson: Transformation is less a technology problem than a systems problem — and solving it requires aligning people, process, and platforms around the uninterrupted movement of value.

Scaling AI Beyond the Proof of Concept

For many enterprises, AI is still a pilot program. A promising prototype here, a promising use case there—but the operational scale that would actually move the needle remains elusive. Brian Moore, a leader at RTX (Raytheon Technologies), one of the world’s largest aerospace and defense firms, has seen this pattern up close and believes the answer lies in physics as much as it does in technology.

Drawing on Industrial DevOps, SAFe, and the constructal law—a principle from physics that describes how flow systems evolve to move things more efficiently—Moore argues that enterprises must architect their AI deployments the way nature architects rivers: with channels deliberately designed to reduce resistance and maximize throughput.

“Realizing the promise of AI in an enterprise requires moving beyond pilots to large-scale deployments, guided by principles derived from the constructal law of physics,” says Moore. “The practices of Industrial DevOps and SAFe align with constructal theory and give us direct guidance on how to rapidly scale AI benefits—and the case studies from RTX prove it’s possible.”

The lesson: Flow isn’t accidental. It has to be engineered.

When Adoption Isn’t Enough

Even where software is deployed at scale, value doesn’t automatically follow. Giso van der Heide, a customer success strategist at Siemens, points to a startling statistic: An estimated 46% of enterprise applications remain underutilized or unused, representing billions of dollars in wasted licenses every year. The culprit, he argues, isn’t bad software — it’s the wrong success metric.

Van der Heide’s concept of Outcome Engineering reframes customer success entirely. Rather than measuring whether users log in, it asks whether they are achieving the specific business outcomes the software was purchased to deliver. His Persona Outcome Fabric maps the Jobs to Be Done for individual user roles and connects them directly to quantified performance measures and ROI—bridging the gap between a product’s features and the measurable value a business actually needs.

AI plays a central role in this model, helping generate customized adoption roadmaps and persona-specific success plans at a scale no human team could manage manually.

“Adoption without clear business impact is merely noise,” says van der Heide. “True customer success requires a structured methodology that measures tactical KPIs alongside actual business impact—and AI-driven components are what make that personalization scalable.”

The Friction at the Intersections

Scaling individual value streams is challenging enough. But as Ja’Mesa Dixon, a flow and agility practitioner at Zayo Group, will tell you, the real complexity begins when those streams start to interact. Shared platforms, competing funding priorities, cross-team dependencies — these intersections are where organizational momentum goes to die.

Dixon’s work helping enterprises evolve from team-level agility to system-level flow has given her an unflinching view of what actually breaks down and what actually helps. Her insight: the solution isn’t heavier governance. It’s smarter coordination — lightweight mechanisms that resolve dependencies without creating bureaucratic drag, and enough psychological safety for teams to surface problems before they compound.

Critically, Dixon emphasizes that Value Stream Networks are as much a human challenge as a structural one. Progress is rarely linear, and leadership’s appetite for speed must be balanced with the reality that trust is built through small, visible wins over time.

“Success in Value Stream Networks isn’t just about frameworks,” says Dixon. “It’s about meeting people where they are, managing the friction at the intersections, and building momentum through visible progress — not above it, but right in the middle of it.”

Transformation is a People Problem

Nowhere is the human dimension of transformation more visible than in talent strategy. TaQuonda Hill, who has led Delta Air Lines’ largest cloud migration and workforce enablement initiatives, has built a blueprint for what she calls a product-centric Cloud Target Operating Model—a framework that treats people, process, and technology as a single integrated system rather than parallel workstreams.

Her experience at Delta underscores a truth that too many transformation programs overlook: Agility at scale requires not just restructured processes but re-energized people. Talent empowerment, inclusive design, and continuous learning aren’t soft add-ons to a technical program—they are the operating conditions that make everything else work.

“True transformation lies at the intersection of technology, people, and process,” says Hill. “Visionary leadership has to translate enterprise strategy into tangible value by aligning technology enablement with people-centric design—and that means building a resilient, future-ready workforce, not just deploying new tools.”

The Common Thread: Flow

What connects Brian Moore’s physics-inspired AI scaling model, Giso van der Heide’s outcome engineering framework, Ja’Mesa Dixon’s Value Stream Network field notes, and TaQuonda Hill’s talent transformation blueprint? All four are, at their core, about the same thing: Removing the barriers that prevent value from moving through an organization — from strategy to execution, from technology to people, from pilot to production.

Flow is not a buzzword. It is a discipline. And the enterprises that master it will be the ones that turn transformation ambition into transformation results.

Hear Brian Moore, Giso van der Heide, Ja’Mesa Dixon, TaQuonda Hill, and many more at Flowtopia Live. Join leaders from across the industry for a day of candid conversation, practical frameworks, and real-world case studies on building enterprise-scale flow.

Join us on June 24th and get flowing.

Exclusive Offer: As part of the TechStrong community, get a free LIVE pass.

Use Code: TECHSTRONGFLOW

Get your Live Pass to Flowtopia Live 2026 here.

June 12, 2026
Break Open the Black Box of Software Development

If you asked an airline executive what their biggest expense was just a few decades ago, they would’ve easily answered, “Jet fuel.” These days, you’d be hard-pressed to find an executive in any industry that wouldn’t answer with, “Software.” But here’s the difference: With jet fuel, that executive would have told you that they can fly x number of seat miles per gallon/liter of fuel (or pounds of fuel, if we’re being technical). With software development, the expense is becoming more material–and far more variable–but the return on that investment is rarely clear. Over the past decade or more, software development has readily assumed its position as the fastest-growing (and often largest) expense line item in virtually every industry. All companies are adding software experiences to or around their products, and they need the people and tools to build, support and continuously optimize those experiences. And that costs money–a lot of money.

The problem is that the language, the metrics and the world in which software development exists do not translate into operational or financial terms that business leaders can readily understand.

“We invested X million dollars into software development last year,” a CFO might say. “And what do we have to show for it?”

Meanwhile, a technology leader might respond with, “We’re doing X deploys a day!” or, “We wrote X thousand lines of code!”

They might as well be speaking different languages because, in practical terms, they are.

And the problem is, that you can’t manage what you can’t measure. To continue our airline theme, software development continues to be a ‘black box’—this hidden-in-plain-sight source of information—that financial executives can’t seem to peek into.

Why the Black Box?

It’s not likely that technology leaders are being purposefully elusive toward business leaders–it’s more likely that they simply are not aligned on what, fundamentally, they need to do to speak the same language.

The question at hand is, actually, “Are these investments we’re making turning into our desired business outcomes?” But the question technology leaders feel compelled to answer is, “What do your people do all day?”

These are questions that Agile and DevOps don’t solve–translating Agile or DevOps metrics (also known as DORA metrics) into actual, actionable metrics for business leaders to understand. Although these metrics are tremendously useful for teams, using them to steer the business is a mistake.

Misaligned in purpose and using different language, business leaders and technology leaders talk past each other as software development costs continue to rise.

But the cost of software development is too high for business leaders to continue to tolerate this ‘black box’ of ever-rising costs. It’s the fiduciary responsibility of financial leaders to understand the value of software investments and figure out how they are affecting business outcomes–or their jobs could be on the line.

In case this seems like a distant threat, it isn’t: I saw a headline recently about a prominent automobile executive who lost their job because they weren’t able to gain the necessary insight into, and business outcomes out of, their significant software development costs and that was affecting the competitiveness of their entire company.

It’s happening. And it’s on both business and technology leaders to break that black box open and start speaking the same language.

The next question, of course, is, “How?”

How to Bridge the Communication Gap Between Technology and Business Leaders

For far too long, business leaders have made do with the activity metrics provided by technology leaders–they’ve tolerated this lack of visibility and found other ‘strategic’ ways to justify the rising software development costs. Perhaps they’ve been questioning how A leads to B–how an increase in funding for software development might result in greater customer retention or app usage, for example–but they have not had a tangible way to prove to themselves, much less to investors, whether this is actually happening.

In order to understand and quantify software development costs, we have to start demanding more from our technology leaders: Working with them to establish realistic objectives and key results (OKRs), and holding them accountable to business metrics that actually relate to outcomes. That means insisting on business results–and not accepting activity metrics in their stead.

Stable-Fund Products, Not Projects

Here’s a hard pill for finance to swallow: When it comes to delivering software-enabled customer experiences, it’s not about optimizing costs. It’s much more nuanced than X dollars in equals X dollars out. Funding by projects might help make the numbers neat, but it doesn’t reflect the reality of the business.

Project-based funding also tends to keep project leaders in a cost-center mentality: It makes it seem as though all resources are fungible; as if the work and the people performing it are interchangeable.

One of the goals of Agile is to maximize value for the customer. But when we fund by project, we often stop funding a piece of work–declaring it done–before it actually starts generating value. We have this false end date that doesn’t reflect the actual ending of the value that we funded to create.

Instead, finance organizations should move to the stable funding of products: Funding an outcome and giving teams the ability to stop, change or extend their work to achieve the business outcomes of the product.

This is a shift that one prominent UK bank has been going through in recent years–from traditional portfolio management to lean portfolio management, including the shift to stable-funding projects. You can learn more about how NatWest Group (formerly Royal Bank of Scotland) has made this shift in this webinar.

Visualize the Flow of Value with Flow Metrics

Finally, we have to stop treating the collection of technology metrics and business metrics as two separate, unrelated efforts. Agile and DevOps are great at accelerating the throughput of your software development … but vanity metrics around how quickly teams are delivering are not enough to justify the growing and material investment in software development.

Finance leaders have the opportunity to be the Babelfish between technology leaders and business leaders. As mentioned above, one way to encourage clearer communication between teams is to use OKRs to naturally tie software development objectives to tangible business results. The more we practice drawing connections between technology and business metrics–the closer we get to finally breaking open that elusive black box.

A key part of this is communicating operational metrics differently: Developing a core set of metrics for tracking the flow of business value in software delivery, and providing a mechanism that correlates the investment in flow for each product’s value stream with the business results for that value stream.

Visualizing the flow of value in a software development process may not be as easy as watching value flow on a manufacturing floor–but that doesn’t mean it’s impossible. Thanks to the vast number of data and visualization tools at our disposal, it is possible to make software products and their value streams just as visible as production lines.

The Flow Framework, created by Dr. Mik Kersten, addresses the pivotal challenge of breaking open the black box of software development and bridging the gap between business and technology to optimize the flow of business value to customers.

Break Open the Black Box with Value Stream Management

The black box of software development is, at its core, a communication issue. It’s on both business and technology leaders to bridge this communication gap. Business leaders need to demand more from technology leaders and insist on translating activity into business metrics. Technology leaders need to practice tying their efforts to tangible business outcomes to help justify their growing expenses.

Of course, it’s helpful to have tools purpose-built to facilitate this visibility and communication.

November 2, 2022
Latest DORA Report From Google Surfaces Raft of DevOps Challenges

The annual Accelerate State of DevOps 2022 report published today by the DevOps Research and Assessment (DORA) team at Google found the percentage of high performers is at a four-year low, with the percentage of low performers rising dramatically from 7% in 2021 to 19% in 2022.

Based on a survey of 1,350 DevOps professionals, the report evaluated organizations based on the five DORA metrics: Deployment frequency, lead time for changes, time to restore service, change failure rate and operational performance. The DORA team also identified clusters of DevOps teams ranging from “starters” to “flowing” based on their overall maturity. Only 17% of respondents achieved a ‘flow’ state defined by high reliability, high stability and high throughput characteristics.

The DORA survey sample shifted this year to include more respondents that are earlier in their careers than in previous reports. However, Claire Peters, DORA research lead at Google, said that while there are a lot of factors that impact those metrics, the current hypothesis is that the COVID-19 pandemic has taken a toll on DevOps productivity as more DevOps professionals continue to work remotely.

The report noted a marked shift in deployment targets requiring new skills, with 54% of respondents now working with container artifacts.

Overall, the report finds software delivery performance is beneficial to organizational performance when operational performance is also high. The challenge is the number of organizations that have achieved high operational performance is relatively small. In fact, the report also noted that site reliability engineering (SRE) practices had a negative impact on software delivery performance until an organization achieved a high level of SRE maturity.

The report also concluded that organizations that built software on and for the cloud tended to have 1.4 times higher organizational performance than those that didn’t. The percentage of respondents using public clouds is 76%, up from 56% in 2021, while the number of respondents not using a cloud stands at 10.5%. Usage of multiple public clouds is at 26%, while 35% reported using a private cloud.

Less clear is the impact on DevOps teams of shifting application security left. The report suggested the biggest challenge organizations face when it comes to achieving application security have more to do with cultural issues than any technical shortcomings. The report notes that organizations that focus on accelerating software delivery without implementing meaningful DevSecOps best practices find themselves in a vicious counterproductive cycle that results in applications being successfully deployed in production environments less often.

Peters noted that there is greater developer fatigue because organizations attempt to address security issues either just before an application is deployed or they remediate applications after they are deployed. Organizations that have low levels of security practices are 1.4 times more likely to experience developer burnout.

The DORA team used both the supply-chain levels for secure artifacts (SLSA) framework defined by Google and the Secure Software Development Framework (SSDF) defined by the National Institute of Standards (NIST) to evaluate organizations. The most widely-adopted practice is application security scanning within a continuous integration/continuous delivery (CI/CD) system, with 63% of respondents reporting these tools are “very” or “completely” established. Preserving code history and using build scripts are also highly established, the report finds.

In general, Peters said the report suggested the biggest predictor of application security success was whether an organization had a high-trust, low-blame culture focused on performance. Organizations that have these cultures tend to have higher organizational performance. Similarly, organizations with teams that felt supported through funding and leadership sponsorships tended to have higher organizational performance, the report noted. In addition, team stability and positive perceptions about one’s team also tended to lead to higher levels of organizational performance. Lastly, companies that offer flexible work arrangements tended to see higher levels of organizational performance.

Each organization, as always, will implement DevOps best practices in ways that best fit its internal culture. However, the latest DORA report makes it clear that the more empowered a DevOps team is the more proficient they tend to become.

September 28, 2022
Value Stream Management: Treat Your Pipeline as Your Most Important Product
When it comes to accelerating customer value, it’s dangerous to focus only on the development and delivery of a digital product. For many enterprises, most time is invested before a single line of code has been written. That explains the recent focus on value stream management (VSM)—it’s forcing us to think beyond Agile and DevOps and consider the whole system, not just one stage in a complex process.

By extending our visibility further upstream where work originates, we can begin measuring the value of creating and protect workflows end-to-end to meet business outcomes sooner. Thinking about how software delivery work moves between all teams in the value stream from request to operation leads us to the significant realization that our tooling pipeline is our most important product and we should treat it as such.

The Rise of Value Stream Management

The growth of value stream management in software delivery has emerged because Agile and DevOps are not enough to remain competitive against the breakneck speeds of the tech oligopoly. The progressively maturing practice takes a systematic approach to continuously measuring and improving end-to-end flow to accelerate time-to-value. Consequently, new tooling is emerging as we think end-to-end in how we collaborate and work faster together—and measure our productivity and effectiveness.

Measuring and understanding end-to-end flow isn’t easy. Software delivery is highly technical and creative knowledge work that can span thousands (or tens of thousands) of IT practitioners. Information is scattered across heterogenous toolchains comprising discipline-specific tools. In fact, some 42% of large-scale software delivery organizations use four to 10 core collaboration tools for portfolio management, design, enterprise agile planning, testing and ITSM, and dozens of additional tools to support the CI/CD pipeline and operations.

Mik Kersten, “Mining the Ground Truth of Enterprise Toolchains, IEEE (Volume 35, Issue 3, May/June 2018, p12-17

Measuring flow, therefore, involves tracing constituent units of value creation and protection as they traverse all team tooling in the form of artifacts such as activities, capabilities, epics, features, stories, bugs and incidents. Yet for too long we’ve been too focused on “the right side” of the value stream, thinking about how products are built and delivered through Agile and DevOps tools, without visibility on how they’re requested, analyzed, processed and designed. Our ability to support customers is hindered because we don’t know where and why work is slowing down outside of development. Our product pipeline is broken, our visibility fractured.

A ‘Broken’ Product Pipeline

Traditionally, CI/CD tools include source code management, build, deploy, quality management and service management. Moving more to the right, you have release management and service management. But too often organizations have ignored integrating tools further to the left such as planning and portfolio management, product management, requirements management and risk management (static and dynamic security scanning tools). This impacts our ability to see and measure end-to-end flow.

In”Accelerate,” lead time is defined as the portion of the value stream from (last) code commit until deployment. While this is essential to improve, it only covers the time from the build being produced through deployment. We need to account for the time that elapses before it hits a developer’s backlog. To shift IT from a project to a product focus, you need to reduce end-to-end flow time by measuring earlier in the delivery life cycle by “shifting left.” As such, you need tooling across the whole value stream through the ideate, create, release and operate phases. A focus only on optimizing the CI/CD tooling and practices is too narrow a view that produces diminishing returns over time.

The Great Disconnect Between IT and the Business

Because of this lack of a holistic view, IT teams don’t feel like they are able to influence the business side of the house, where most of the work in progress (WIP) starts, such as requests for features or application changes. These intake processes many times result in teams taking on too much WIP and neglecting work. Many teams need to “stop starting” and to “start finishing” to deliver business value more quickly. Crucially, this is the area where business leaders have their own processes and where tools are less likely to be used; it’s a world dominated by spreadsheets, documents and PowerPoint presentations. This is where work is least visible and measurable. And this is precisely where we need VSM: to turn the light on starting with the customer needs so we can continuously improve end-to-end flow. This requires a culture shift from both IT and business/product owners. And that requires taking a broader, more holistic view of the tools and practices in the business space.

Architecting for Flow

An end-to-end integrated pipeline product can be architected for flow. For example, tools used for visual system management (e.g. Jira, Azure DevOps) are becoming more of a commodity with the real differentiation becoming how they can be integrated with tools used for portfolio, defect and service management, since those are the sources of work that will flow into product-based teams for development of business value and unplanned work. Teams should think about how they can work more closely with business portfolio teams on continuous flow (and funding) because eventually this work will become the next bottleneck that needs to be addressed on the journey to accelerate business delivery capability.

Keep an Eye on Value Protection Work

With end-to-end visibility into work across the pipeline, you can abstract that connected technical data into overarching value stream metrics—known as flow metrics—to identify areas where to run experiments to improve end-to-end flow. Here we mustn’t just focus on work that creates value but protects it, too.

We should provide our software delivery teams with the psychological safety to devote time to address debt and neglected work. Debt creates a drag on feature velocity because changes become more complex and take longer to make. In addition, with higher technical debt, more time is spent on addressing defects, which starves the capacity to perform feature work. As debt builds, the following is observed:
- Flow velocity decreases.
- Flow time increases.
- Quality decreases as the code base becomes more fragile/costs increase.
All of these result in reduced happiness for both the customer and the team, increasing the risk of team burnout. There are many cautionary tales of this occurring where market leaders ended up losing considerable market share due to inattention to technical debt. These stories exemplify why it is necessary to observe flow distribution to ensure that prioritization decisions made with the product owners, include providing necessary capacity to address technical debt.

Applying the concepts of value stream architecture to your end-to-end pipeline is the key differentiator for organizations that are trying to make it through disruptive periods such as today, as COVID-19 widens the gap between those who have mastered software at scale and are thriving and the many enterprises struggling to pivot and adapt in the digital-first climate.
November 5, 2020
Does Agile Methodologies Work for IT Infra?

Recently, I had a fascinating conversation with my old friend Joe–the name has been changed but the conversation is real and unedited–an IT Infrastructure program manager from a large IT service provider.

After a bit of chit-chat on weather and football, we started talking about infrastructure in a typical large enterprise.

Joe: I am not sure how agile applies to infra space, I can understand it works well for software development, but not in IT Infra space.

Me: OK, why do you think so?

Joe: Well, we have lots of coupling between components, infra projects tend to be big migration or upgrade moves and highly risky. I know many large enterprises in Europe that we work for and many of them haven’t looked at agile in the infra space.

Forty minutes and couple of beers later…

Joe: I have to send four reports every Friday afternoon on status, risks, issues.

Me: Why do you need so many reports?

Joe: I can’t share all the same info to the client that I share internally. There are some risks that we manage internally. My client wants the info on Jira and confluence. My regional manager wants it on an internal tool (some five-word acronym that I forget!). My account manager wants it as a PPT, as this is an important account and he says this needs to be tracked closely. My global manager wants it on a different tool, inherited from a previous merger as my employer was recently acquired by another service provider.

Basically, most of my Friday is spent copy pasting information across these reports. Then I spend most of Mondays reconciling the numbers and responding to follow-on queries, and Tuesdays justifying the cost of resources, change in resource costs because many projects are usually in Amber or Red. I think it usually works well if I have a good PMO.

Me: How many people do you have on the current programme?

Joe: We have four PMs and seven infra resources on this infra upgrade and migration programme. There is a different provider who looks at the application changes and they have a team of managers and developers etc. The seven infra resources I referred to are also busy with BAU work.

Me: So, you have seven part time infra people who are the actual doers, one programme manager, four PMs and PMO, client side vendor manager(s) and you are running half a dozen projects.

Joe: Yes, as you can see, we can’t be agile in infra.

Me: Why do you need projects? And, why do you need so many of them?

Joe: What? No projects? Surely, that’s how it is always done! How do we ensure control, governance?

— Aditya Vadaganadam

September 18, 2019
DevOps Best Practices: Break Down Silos, Avoid Drift and Optimize for Flow
In organizations trying to achieve agility with stability and quality, silos (of teams, tools, and processes) create major stumbling blocks to managing configuration drift between pre-production and production systems. Breaking down those silos, improving visibility into configuration drift and facilitating collaboration for improved flow are the keys to overcoming this challenge.

The challenge of staying in sync when both sides are moving

Production environments aren’t static. Whether you are applying patches for improved security, optimizing settings for availability and performance, or making changes for a technology refresh, production systems are continuously evolving. As new releases are being developed, tested, and staged, pre-production environments are continuously evolving too. Ideally you want to develop with production in mind, but that gets harder and harder when both sides are changing and different teams are responsible for different environments and live in their “own worlds.” (e.g. the common split between QA teams, release teams and production operations).

Even within pre-production, staying in sync is a significant challenge. For example, because applications ride on middleware and middleware settings are unique to each application, keeping systems in sync from QA to staging can be a real challenge.

QA and staging environments are regularly reconfigured to support multiple releases that are in process at the same time. The costly way, often employed to minimize changes, is to “protect” QA environments for a certain application or release leading to low utilization and bottlenecks. The rapid proliferation of mobile applications to front end legacy systems magnifies the challenge by adding another dependency layer.

Disconnected teams, information, and perspectives (those pesky silos)

Why are these challenges so hard to overcome if we know the issues? Many IT teams operate in their “own worlds” with tailor-made but disconnected tools and information resources. Silos have evolved because they permit the specialization required to deal with the immense, growing complexity IT deals with everyday, but they have now become the biggest impediment to agility and quality.

For example, the operations team has access to systems management alerts that most others don’t, engineers have access to build systems and scripts that others don’t, developers have access to functional requirements and stories that others don’t, and change approvals are stored in a system of record that only certain people (often managers) have access to – just to name a few silos.

Coordinating work and sharing information across these silos of teams often equates to lots of meetings, conference calls, and email threads. This is not the most efficient way to get work done and it’s not particularly effective in sharing information to keep environments in sync.

“Didn’t you see my email from yesterday with the attachment listing all the middleware settings required for today’s test?” “Weren’t you in the meeting three weeks ago when I said a Unix server was probably going to be needed in addition to a Windows machine?” “I told so and so in an instant message on Friday to run this script before he went out on PTO, but he must have forgotten. Who else has root permission to run it while he’s out?”

Sadly, these kinds of statements are the norm, not the exception, in most IT shops.

The reality is there is inefficient coordination to manage all the steps and handoffs between application owners, development, QA, release and production functions. Instead, I recommend thinking about the software development and deployment lifecycle as an “uber change” and “uber collaboration” process that must be viewed and managed from end to end; from business requirements, to development, to QA to staging to production. All of the different silos of teams must have a way to work together in a unified way with access to critical information from upstream and downstream teams to ensure a smooth lifecycle end to end.

Automation ≠ DevOps

The heart of DevOps is collaboration and managing change, yet the conversation often seems to start with automation. The fallacy here is the belief that we can eliminate the need to collaborate and manage change by removing people from the process and relying on the machine. This out of balance perspective between the machine and people creates more problems than it solves.

Automation tools, while important and necessary, are not all that’s needed to truly solve these synchronization and deployment issues. In addition, to have environments only changed by automation tools and never by people during the firefighting of incidents is unrealistic. You need your people armed with intelligence and tribal knowledge to drive your automations, but all too often we rely on individuals with disconnected perspectives and limited awareness so things get missed or overlooked.

Automation does what it’s supposed to do, but it makes a critical assumption; what’s being changed is configured how it’s supposed to be configured. If drift has occurred because critical settings were modified to remediate an incident or compliance issue, without the automation engineer aware of the change, the automated deployment will result in you hearing a loud “boom!”

As one popular DevOps Twitter handle has said, “To make a configuration error is human – to do it across a thousand servers is DevOps.” While that may seem like an extreme statement designed for comedic effect, it isn’t extreme to say that thousands of deployments go south every day because systems aren’t configured the way we expect them to be given that the complex software stacks that multi-tiered applications now depend on are touched in so many ways. Automation alone doesn’t solve the DevOps challenge.

Negative consequences to business

Of course, the issues I’m raising here aren’t just IT issues; they are real business issues. Time to market is typically the number one impacted area. For example, if a two-week development sprint is used to get a business critical need addressed, but then it takes weeks and weeks to get the sprint through the quality cycle and into production due to resource constraints and configuration issues, what have we achieved?

Stability and quality are the next big area of impact. When change is rapidly introduced through new releases and unexpected drift occurs in environments, performance and availability impacts occur. It is one thing for IT to be perceived as slow and unresponsive to the business, but it is quite another when they are perceived as slow and quality is poor resulting in business impact.

Just when IT thinks it can’t get any worse, the business asks a presumably simple question, “What happened?” IT’s typical response, “Let me get back to you on that.” And then begins a scramble of hours, days or weeks as IT’s best people desperately try to figure out what went wrong. This is obviously not the best confidence builder for the business.

Compliance is typically the other casualty as there is often very little awareness of operational policies on the development side of the house so deployment packages and instructions aren’t sensitive to compliance issues. Building and testing an application without full awareness of policy governance across the whole, multi-tier stack is bound to introduce delays and errors when attempting to deploy in production.

Visibility, configuration transparency and collaboration; a better way!
1. Understand drift for the entire application service. Rather than everyone having their own disjointed perspectives on databases, app middleware, network or firewall, and server configurations, provide team members with an understanding from the business service (or application) perspective with direct visibility into the differences between how things should be configured and how they are actually configured for the entire multi-tier application stack. This enables you to create smarter automations and initiate change processes to bring configurations back in line when necessary rather than waiting for a failed deployment to occur, identifying the root cause, and correcting it while the business waits.
2. Optimize for flow not individual processes. As stated in the previous example, making one process faster (e.g. development) without considering its impact on QA, staging and production will generate new constraints downstream. Instead, view DevOps as one continuous process from goals and requirements all the way through to deployment into production and define success for the team in those terms. When you optimize for flow instead of silos, you will be amazed at how teams share more and collaborate better.
The benefits of breaking down silos through better visibility, understanding drift and collaboration are significant:
- faster time to market;
- quicker ability to react to competitive pressures;
- deployment with fewer errors;
- better compliance with policies; and
- greater productivity for everyone.
To sum it up, if you break down silos, identify and resolve drift, and manage for flow (while avoiding islands of automation, information, and people) you can unlock agility, maintain stability and ensure the quality your business demands.
May 16, 2014