Author: Cameron Van Orman

  • Break Open the Black Box of Software Development

    Break Open the Black Box of Software Development

    If you asked an airline executive what their biggest expense was just a few decades ago, they would’ve easily answered, “Jet fuel.” These days, you’d be hard-pressed to find an executive in any industry that wouldn’t answer with, “Software.” But here’s the difference: With jet fuel, that executive would have told you that they can fly x number of seat miles per gallon/liter of fuel (or pounds of fuel, if we’re being technical). With software development, the expense is becoming more material–and far more variable–but the return on that investment is rarely clear. Over the past decade or more, software development has readily assumed its position as the fastest-growing (and often largest) expense line item in virtually every industry. All companies are adding software experiences to or around their products, and they need the people and tools to build, support and continuously optimize those experiences. And that costs money–a lot of money.

    The problem is that the language, the metrics and the world in which software development exists do not translate into operational or financial terms that business leaders can readily understand.

    “We invested X million dollars into software development last year,” a CFO might say. “And what do we have to show for it?”

    Meanwhile, a technology leader might respond with, “We’re doing X deploys a day!” or, “We wrote X thousand lines of code!”

    They might as well be speaking different languages because, in practical terms, they are.

    And the problem is, that you can’t manage what you can’t measure. To continue our airline theme, software development continues to be a ‘black box’—this hidden-in-plain-sight source of information—that financial executives can’t seem to peek into.

    Why the Black Box?

    It’s not likely that technology leaders are being purposefully elusive toward business leaders–it’s more likely that they simply are not aligned on what, fundamentally, they need to do to speak the same language.

    The question at hand is, actually, “Are these investments we’re making turning into our desired business outcomes?” But the question technology leaders feel compelled to answer is, “What do your people do all day?”

    These are questions that Agile and DevOps don’t solve–translating Agile or DevOps metrics (also known as DORA metrics) into actual, actionable metrics for business leaders to understand. Although these metrics are tremendously useful for teams, using them to steer the business is a mistake.

    Misaligned in purpose and using different language, business leaders and technology leaders talk past each other as software development costs continue to rise.

    But the cost of software development is too high for business leaders to continue to tolerate this ‘black box’ of ever-rising costs. It’s the fiduciary responsibility of financial leaders to understand the value of software investments and figure out how they are affecting business outcomes–or their jobs could be on the line.

    In case this seems like a distant threat, it isn’t: I saw a headline recently about a prominent automobile executive who lost their job because they weren’t able to gain the necessary insight into, and business outcomes out of, their significant software development costs and that was affecting the competitiveness of their entire company.

    It’s happening. And it’s on both business and technology leaders to break that black box open and start speaking the same language.

    The next question, of course, is, “How?”

    How to Bridge the Communication Gap Between Technology and Business Leaders

    For far too long, business leaders have made do with the activity metrics provided by technology leaders–they’ve tolerated this lack of visibility and found other ‘strategic’ ways to justify the rising software development costs. Perhaps they’ve been questioning how A leads to B–how an increase in funding for software development might result in greater customer retention or app usage, for example–but they have not had a tangible way to prove to themselves, much less to investors, whether this is actually happening.

    In order to understand and quantify software development costs, we have to start demanding more from our technology leaders: Working with them to establish realistic objectives and key results (OKRs), and holding them accountable to business metrics that actually relate to outcomes. That means insisting on business results–and not accepting activity metrics in their stead.

    Stable-Fund Products, Not Projects

    Here’s a hard pill for finance to swallow: When it comes to delivering software-enabled customer experiences, it’s not about optimizing costs. It’s much more nuanced than X dollars in equals X dollars out. Funding by projects might help make the numbers neat, but it doesn’t reflect the reality of the business.

    Project-based funding also tends to keep project leaders in a cost-center mentality: It makes it seem as though all resources are fungible; as if the work and the people performing it are interchangeable.

    One of the goals of Agile is to maximize value for the customer. But when we fund by project, we often stop funding a piece of work–declaring it done–before it actually starts generating value. We have this false end date that doesn’t reflect the actual ending of the value that we funded to create.

    Instead, finance organizations should move to the stable funding of products: Funding an outcome and giving teams the ability to stop, change or extend their work to achieve the business outcomes of the product.

    This is a shift that one prominent UK bank has been going through in recent years–from traditional portfolio management to lean portfolio management, including the shift to stable-funding projects. You can learn more about how NatWest Group (formerly Royal Bank of Scotland) has made this shift in this webinar.

    Visualize the Flow of Value with Flow Metrics

    Finally, we have to stop treating the collection of technology metrics and business metrics as two separate, unrelated efforts. Agile and DevOps are great at accelerating the throughput of your software development … but vanity metrics around how quickly teams are delivering are not enough to justify the growing and material investment in software development.

    Finance leaders have the opportunity to be the Babelfish between technology leaders and business leaders. As mentioned above, one way to encourage clearer communication between teams is to use OKRs to naturally tie software development objectives to tangible business results. The more we practice drawing connections between technology and business metrics–the closer we get to finally breaking open that elusive black box.

    A key part of this is communicating operational metrics differently: Developing a core set of metrics for tracking the flow of business value in software delivery, and providing a mechanism that correlates the investment in flow for each product’s value stream with the business results for that value stream.

    Visualizing the flow of value in a software development process may not be as easy as watching value flow on a manufacturing floor–but that doesn’t mean it’s impossible. Thanks to the vast number of data and visualization tools at our disposal, it is possible to make software products and their value streams just as visible as production lines.

    The Flow Framework, created by Dr. Mik Kersten, addresses the pivotal challenge of breaking open the black box of software development and bridging the gap between business and technology to optimize the flow of business value to customers.

    Break Open the Black Box with Value Stream Management

    The black box of software development is, at its core, a communication issue. It’s on both business and technology leaders to bridge this communication gap. Business leaders need to demand more from technology leaders and insist on translating activity into business metrics. Technology leaders need to practice tying their efforts to tangible business outcomes to help justify their growing expenses.

    Of course, it’s helpful to have tools purpose-built to facilitate this visibility and communication.

  • When AI Met DevOps: Machine Teaching

    When AI Met DevOps: Machine Teaching

    One of my favorite quotes about artificial intelligence (AI) isn’t from a data scientist or tech industry analyst. It’s from a doctor.

    When asked if AI would eventually replace radiologists, Dr. Curtis Langoltz of Stanford University pithily replied “No, but radiologists who use AI will replace those who do not use it.”

    This great insight is highly applicable to IT. Yes, there are many ways we can use AI methods to fully automate a broad range of DevOps tasks. Companies, including CA Technologies, are actively involved in the development of solutions that enable our customers to do exactly that.

    Algorithmic machine learning, however, doesn’t just empower systems to perform tasks and solve problems autonomously. It also makes them great active partners with human beings. In fact, much of what machines learn they also wind up teaching.

    The synergy between AI and human radiologists, for example, stems in part from the fact that digital systems can differentiate about 200 levels of gray in a diagnostic image—compared to only about 16-20 that are discernable by the human eye. Train an AI system with enough images, and that precise cognitive power can more effectively detect that something is going on.

    But for the most effective diagnostic process, you don’t just depend on that detection alone. You use that detection to empower a human diagnostician who can apply a broad understanding of pathologies and deep experience with the complexities of individual patients to deliver the highest quality care.

    In DevOps, we can do the same. We can use AI to capture insights that teach us how to continuously optimize our workflows and processes. We can also use our AI learnings to push our work up higher on the value chain.

    More specifically, the synergy between AI and human intellect can:

    Make development smarter. The speed, quality, and efficiency of development pipelines can be affected by all kinds of subtle factors. A less-than-optimally designed API, for example, can be a small but chronic stumbling block to everyone who has to use it. Scrum outcomes can be undermined by anything from a particular type of technical challenge to a nascent personality conflict.

    By capturing a rich set of DevOps metrics and applying machine learning to those metrics, development leaders can discover process bottlenecks and skilling shortfalls. They can better coach individuals and promote team collaboration. The result: a better working environment that facilitates digital agility for the enterprise and higher satisfaction/retention for valuable employees.

    Make ops smarter. Enterprises are running increasingly volatile and complex workloads on increasingly hybridized infrastructure. At the same time, the tolerance of internal and external users for latency and outages continues to approach zero. There are also real costs associated with performance problems.

    The elastic capacity of public and private cloud does much to help with workload volatility. But adding cloud capacity also has its costs—and end-to-end application performance often depends on back-end systems that are not cloud-based. So not every performance issue can be solved by simply throwing more capacity at it. Nor should it be, if a rearchitecting can fix a bottleneck less expensively.

    Here again, AI can teach us a lot. We can uncover opaque interdependencies in processing load and data throughput. We can spot conditions when it may make business sense to throttle cloud costs that aren’t cost-justified. We can even better understand the real-world conditions—whether patterns in customer behaviors or our own marketing programs—that are driving our demand spikes and troughs. All of this helps us deliver consistently responsive digital experiences at a cost that makes good business sense.

    Make security smarter. AI is already being broadly implemented in security solutions such as endpoint protection and threat response to automate the detection and neutralization of anomalous activities in the enterprise environment. But effective multi-layer security isn’t just about finding and stopping exploits. It’s also about building applications that are themselves inherently less vulnerable to hacking. This is the essence of DevSecOps.

    AI has huge potential value here. We are writing a rapidly growing volume of increasingly sophisticated code. It is very easy for subtle vulnerabilities to hide in that code. As our development practices become more complex—often including multiple contractors—it becomes more difficult to understand exactly where and why these vulnerabilities were introduced into our code. Machine learning can teach us the answers to these questions, so we can more proactively secure our data and our businesses.

    It’s especially interesting to consider what may happen as we start to apply AI to DevSecOps across our organizations, as well as within them. More diverse inputs enable machine learning to discover more factors that impact code pipeline performance. By aggregating our knowledge about how we build, deliver and secure our code, we are all likely to benefit with better practices and stronger guardrails.

    — Cameron Van Orman