devopsy.kinsta.cloud

Tag: McKinsey

  • Should You Measure Developer Productivity?

    Should You Measure Developer Productivity?

    A recent McKinsey article entitled ‘Yes, you can measure software developer productivity‘ has got the software developer community talking. And the general consensus is they’re not happy.

    Most software developers are not enthusiastic about the prospect of having their individual performance metrics monitored. “Anyone who attempts to measure software in some sort of scientific way is just doing the wrong thing,” said YouTube personality ThePrimeTime. Dave Farley of Continuous Delivery even goes as far as to call the McKinsey report “harmful dribble.”

    This debate arises as developer productivity engineering (DPE) is gaining traction as a means to improve developer performances and, perhaps, in response to employment landscape shifts like the Great Resignation, which saw developers and DevOps pros leaving their roles. Companies like Spotify, LinkedIn, Netflix and others are also investing in internal developer platforms (IDPs) to increase developer productivity and satisfaction amid a limited talent supply.

    Alongside these ambitions, much has been written about how managers can measure developer productivity in terms of qualitative metrics, like developer satisfaction, as well as quantitative metrics, like number of commits or story points, deployment frequency or failure rates. But can tracking such metrics deliver tangible results? Or do they go too far?

    Below, we’ll consider whether you can actually measure software developer productivity. But more than that, we’ll evaluate whether this is a meaningful tactic with desirable outcomes.

    Ways to Measure Development Productivity: DORA

    First, let’s look at some of the standard techniques to measure development productivity. The DORA metrics, produced by the DevOps Research and Assessment (DORA) team at Google after years of research, identified four key metrics:

    1. Deployment frequency: How often you ship changes.
    2. Lead time for changes: The time between a commit and it reaching production.
    3. Change failure rate: The percent of releases that fail; downtime, worsened service or rollbacks.
    4. Time to restore service: The average time it takes to respond to an issue, also known as mean-time-to-recovery.

    These metrics could be helpful indicators of a software development team’s overall agility and health. They can signal bottlenecks in the CI/CD process or highlight where too much time is spent remediating issues, encouraging positive changes. Solving these root causes could all improve a team’s productivity.

    Going Beyond DORA: SPACE

    DORA also added reliability to the list. That’s understandable, given the interest in building reliable systems. But things start to get a little more complex when SPACE enters the equation.

    Proposed in a 2021 paper by GitHub researchers, SPACE augments DORA with the following attributes:

    • Satisfaction and well-being
    • Performance
    • Activity
    • Communication and collaboration
    • Efficiency and flow

    Metrics like satisfaction or communication and collaboration are pretty subjective. Plus, it can be challenging to gather this data and encourage engineers to speak honestly in surveys. Measuring the activity of individual members, like the number of issued closed or pull requests made, isn’t exactly tied to end value and could even produce a negative psychological effect.

    “Consequently, teams can become fixated on them, as seemingly positive changes are relatively easy to achieve,” wrote James Walker. He adds that SPACE still doesn’t tell you why measurements have value—furthermore, it lacks actionable information and might be tricky to implement, given developer pushback against performance analysis systems.

    Opportunity-Focused Metrics

    The McKinsey report adds to the mix what it calls “opportunity-focused productivity metrics,” which include inner/outer loop time spent, a Developer Velocity Index benchmark, contribution analysis and a talent capability score. (It’s not totally clear how these metrics can be computed). It then organizes all these metrics into a framework that further segments measurements into system, team and individual levels.

    McKinsey's framework for measuring developer productivity

    While some believe in the framework’s legitimacy, many commentators don’t see the practicality of implementing such a framework in practice. “Introducing the kind of framework that McKinsey is proposing is wrong-headed and certain to backfire,” wrote the PragmaticEngineer.com. “The report is so absurd and naive that it makes no sense to critique it in detail,” wrote Ken Beck.

    Things to Avoid With Developer Productivity Metrics

    Apparently, developers are very weary of being measured. And given the recent backlash, it seems like overreliance on productivity metrics could negatively impact the engineering culture. So, engineering leadership should be careful when implementing these sorts of tactics. Here are some anti-patterns to consider.

    Don’t make it about individual performances. Metrics like DORA have become standards for measuring the efficiency of software teams, but they shouldn’t be applied at the individual level. “Measuring software development in terms of individual developer productivity is a terrible idea,” said Dave Farley. Technology development is more of a collaborative process; performance metrics should reflect how the team operates.

    Don’t measure developer productivity in isolation. On that note, performance metrics shouldn’t be judged in isolation. Issues with failures, for example, often depend on contexts outside of an individual developer’s control. At the same time, it can be hard to correlate individual developer contributions to end customer satisfaction.

    Don’t create an opportunity to gamify productivity systems. Leadership shouldn’t directly tie a developer’s salary, reputation or position to whether or not they hit certain metrics. If productivity systems carry this much weight, they could become gamefied.

    Don’t put in more effort than it’s worth. In tech, where there’s hype, there’s money to be made—and a host of vendors ready to hawk monitoring tools. What’s more, to enact these productivity measurement enhancements, McKinsey noted that “most companies’ tech stacks will require potentially extensive reconfiguration.” Devoting many engineering resources to monitor engineers just seems like a big pill to swallow.

    Don’t put too much faith in metrics. Lastly, metrics only show one side of the picture. They are only data points and can’t always depict what it’s really like on the ground. Furthermore, qualitative responses also might be skewed due to reluctance to speak up.

    Remember Goodhart’s Law and Focus On Quality

    Personally, I believe that measuring things like developer productivity, efficiency, experience and satisfaction does have some merit for some businesses. Most teams are now working asynchronously, and metrics help leaders take the pulse of a team’s performance, even if they are all remote workers.

    It’s also nothing new. Leadership has been attempting to measure developer productivity for years. But, these metrics often lack context and come with many caveats to consider. Thus, they shouldn’t be given a huge seat at the table.

    I like this ‘law’ from British economist Charles Goodhart, which is colloquially summarized as “when a measure becomes a target, it ceases to be a good measure.” In other words, when you start working for the metrics, the engineering culture and overall code quality might decline. This matters, since a separate study found that increases in code quality tend to be followed by an increase in developer productivity.

    So, focus first on establishing code quality and, hopefully, productivity will flow.

  • Internal Developer Platform (IDP) Reference Architectures

    Internal Developer Platform (IDP) Reference Architectures

    If you’ve been following the DevOps space recently, you’ve probably noticed the excitement brewing around platform engineering. Despite uncertainty in other areas of tech, it seems like platform engineering is poised for growth—Gartner even expects that by 2026, 80% of software engineering organizations will have established platform teams.

    The platform engineering idea hinges on an internal tooling system known as an internal developer platform (IDP), which platform teams own and operate and is exposed to developers to improve their efficiency, satisfaction and release fluidity. IDPs have been the talk of the town. But what does an IDP actually look like? What sort of components are they usually composed of?

    Well, at PlatformCon 2023, McKinsey representatives shared a reference architecture for building out IDPs. This framework was modeled after hundreds of internal platforms and could be a helpful benchmark for other organizations as they begin developing and standardizing their internal developer platforms.

    Below, we’ll examine the sample IDP reference architecture and its components. We’ll also have a refresher on the platform engineering concept and offer some expert tips on overcoming potential obstacles while implementing platform engineering and IDPs.

    Why Engineer Internal Platforms, Why Now?

    First, for those new to the idea, let’s explain what platform engineering is. I’ve previously defined platform engineering as “maintaining an internal self-service platform that serves the needs of developer consumers.” The practice doesn’t necessarily supplant DevOps, rather, the experts I’ve spoken to view it as the next evolution.

    This trend is essentially a response to developers overseeing more and more aspects of software life cycle management. Complexity in enterprise-level software delivery workflows has escalated in the last decade, leading to what The State of Platform Engineering Report describes as “increased cognitive load on engineers.” This is negative, as it could lead to shadow operations and burnout.

    The fact is, most developers require the same sort of technologies, which could be standardized across the board. For instance, take the consolidation around containers and Kubernetes, infrastructure-as-code and the need to implement similar build configurations and security policies across applications.

    By outsourcing the maintenance of tools for shipping and running cloud-native applications to a common platform, you take the burden off developers and free them up to code. Also, by centralizing DevOps tooling, you can enforce more governance and build out internal standards. Which should help reusability and overall deployment agility … at least, that’s the promise of it all. 😉

    Sample IDP Reference Architectures

    So, let’s turn to some real-world examples to see how this is being implemented. Mike Gatto, senior DevOps engineer, McKinsey, shared an internal platform reference architecture for AWS-specific environments:

    McKinsey’s Internal Developer Platform reference architecture

    As you can see, the architecture is divided into these subgroups. At a high level, they perform the following functions:

    • Developer control plane: This is the version control, where developers publish their code. They provide a workload specification (in McKinsey’s case, a Score file) that codifies specific configurations.
    • Integration and delivery: This includes the CI pipeline to build application code into a container and publish it to a registry. This notifies the platform orchestrator to initiate a build. Then, resources are provisioned. This area also includes a CD pipeline and infrastructure control plane.
    • Monitoring and logging: This plane includes tools for observability, analytics, and application performance monitoring.
    • Security plane: This area includes tools for secrets management, policies, and security vetting.
    • Resource plane: This plane encompasses cluster management, database tools, networking, and messaging architectures.

    “By separating developer and platform team concerns and allowing them to control the respective areas they’re experienced in separately, without requiring too much overlap,” said Gatto, “we’re making deployments more compliant and better aligned with client initiatives.”

    Humanitec has also created internal developer platform reference architectures for GCP and Azure, displayed below.

    IDP Reference Architecture for GCP
    IDP Reference Architecture for Azure

    PlatformEngineering.org has also expanded McKinsey’s original IDP reference architecture with a tooling landscape that adopts the same umbrella subgroupings but includes other pluggable tools.

    The Internal Developer Platform tooling landscape

    Roadblocks to Developer Platforms

    Starting from a blueprint could be a big help to rapidly construct IDPs. Yet, there will still be some potential roadblocks associated with adopting an internal developer experience platform. One of them is creating common platforms that actually cater well to all environments.

    “You can not and should not try to please everybody,” said Kaspar von Grünberg, CEO and co-founder of Humanitech. “Platform engineering is an 80/20 game.”

    “Platform organizations often face a lot of pressure when it comes to meeting demands for building more services that are both agile and agnostic to cloud and data center environments,” said Venkat Ramakrishnan, VP, product management and engineering, Portworx by Pure Storage. “At the same time, almost all platform engineering teams are understaffed while facing the need to support large application development initiatives and developer teams twice their size.”

    Tips to Get Platform Engineering On Track

    Reference architectures for various clouds could help cater IDPs to different scenarios. But outside of that, it will take a product mindset approach to strategically accept feedback and iterate the platform. This will involve doubling down on developer experience and adopting cloud-native foundations that can scale.

    “Platform engineering teams must consider adopting Kubernetes as it enables them to manage and deploy applications in production at scale, without having to manage applications at the individual level,” added Ramakrishnan. “Additionally, it may not be realistic to expand platform engineering teams alongside growing platforms, so teams should also consider leveraging automation and self-service capabilities to help them better manage projects at scale.”

    Is ‘You Build It, You Run It’ Dead?

    The idea of ‘you build it, you run it’ is starting to look a little antiquated next to the new and shiny platform model. It’s tough to say, but I’ll be keeping an eye on this area to see if all the effort to standardize the internal engineering approach pays off!