devopsy.kinsta.cloud

Tag: internal developer platform

Best of 2024: Internal Developer Platform (IDP) Reference Architectures
If you’ve been following the DevOps space recently, you’ve probably noticed the excitement brewing around platform engineering. Despite uncertainty in other areas of tech, it seems like platform engineering is poised for growth—Gartner even expects that by 2026, 80% of software engineering organizations will have established platform teams.

The platform engineering idea hinges on an internal tooling system known as an internal developer platform (IDP), which platform teams own and operate and is exposed to developers to improve their efficiency, satisfaction and release fluidity. IDPs have been the talk of the town. But what does an IDP actually look like? What sort of components are they usually composed of?

Well, at PlatformCon 2023, McKinsey representatives shared a reference architecture for building out IDPs. This framework was modeled after hundreds of internal platforms and could be a helpful benchmark for other organizations as they begin developing and standardizing their internal developer platforms.

Below, we’ll examine the sample IDP reference architecture and its components. We’ll also have a refresher on the platform engineering concept and offer some expert tips on overcoming potential obstacles while implementing platform engineering and IDPs.

Why Engineer Internal Platforms, Why Now?

First, for those new to the idea, let’s explain what platform engineering is. I’ve previously defined platform engineering as “maintaining an internal self-service platform that serves the needs of developer consumers.” The practice doesn’t necessarily supplant DevOps, rather, the experts I’ve spoken to view it as the next evolution.

This trend is essentially a response to developers overseeing more and more aspects of software life cycle management. Complexity in enterprise-level software delivery workflows has escalated in the last decade, leading to what The State of Platform Engineering Report describes as “increased cognitive load on engineers.” This is negative, as it could lead to shadow operations and burnout.

The fact is, most developers require the same sort of technologies, which could be standardized across the board. For instance, take the consolidation around containers and Kubernetes, infrastructure-as-code and the need to implement similar build configurations and security policies across applications.

By outsourcing the maintenance of tools for shipping and running cloud-native applications to a common platform, you take the burden off developers and free them up to code. Also, by centralizing DevOps tooling, you can enforce more governance and build out internal standards. Which should help reusability and overall deployment agility … at least, that’s the promise of it all. 😉

Sample IDP Reference Architectures

So, let’s turn to some real-world examples to see how this is being implemented. Mike Gatto, senior DevOps engineer, McKinsey, shared an internal platform reference architecture for AWS-specific environments:

McKinsey’s Internal Developer Platform reference architecture

As you can see, the architecture is divided into these subgroups. At a high level, they perform the following functions:
- Developer control plane: This is the version control, where developers publish their code. They provide a workload specification (in McKinsey’s case, a Score file) that codifies specific configurations.
- Integration and delivery: This includes the CI pipeline to build application code into a container and publish it to a registry. This notifies the platform orchestrator to initiate a build. Then, resources are provisioned. This area also includes a CD pipeline and infrastructure control plane.
- Monitoring and logging: This plane includes tools for observability, analytics, and application performance monitoring.
- Security plane: This area includes tools for secrets management, policies, and security vetting.
- Resource plane: This plane encompasses cluster management, database tools, networking, and messaging architectures.
“By separating developer and platform team concerns and allowing them to control the respective areas they’re experienced in separately, without requiring too much overlap,” said Gatto, “we’re making deployments more compliant and better aligned with client initiatives.”

Humanitec has also created internal developer platform reference architectures for GCP and Azure, displayed below.

IDP Reference Architecture for GCP

IDP Reference Architecture for Azure

PlatformEngineering.org has also expanded McKinsey’s original IDP reference architecture with a tooling landscape that adopts the same umbrella subgroupings but includes other pluggable tools.

The Internal Developer Platform tooling landscape

Roadblocks to Developer Platforms

Starting from a blueprint could be a big help to rapidly construct IDPs. Yet, there will still be some potential roadblocks associated with adopting an internal developer experience platform. One of them is creating common platforms that actually cater well to all environments.

“You can not and should not try to please everybody,” said Kaspar von Grünberg, CEO and co-founder of Humanitech. “Platform engineering is an 80/20 game.”

“Platform organizations often face a lot of pressure when it comes to meeting demands for building more services that are both agile and agnostic to cloud and data center environments,” said Venkat Ramakrishnan, VP, product management and engineering, Portworx by Pure Storage. “At the same time, almost all platform engineering teams are understaffed while facing the need to support large application development initiatives and developer teams twice their size.”

Tips to Get Platform Engineering On Track

Reference architectures for various clouds could help cater IDPs to different scenarios. But outside of that, it will take a product mindset approach to strategically accept feedback and iterate the platform. This will involve doubling down on developer experience and adopting cloud-native foundations that can scale.

“Platform engineering teams must consider adopting Kubernetes as it enables them to manage and deploy applications in production at scale, without having to manage applications at the individual level,” added Ramakrishnan. “Additionally, it may not be realistic to expand platform engineering teams alongside growing platforms, so teams should also consider leveraging automation and self-service capabilities to help them better manage projects at scale.”

Is ‘You Build It, You Run It’ Dead?

The idea of ‘you build it, you run it’ is starting to look a little antiquated next to the new and shiny platform model. It’s tough to say, but I’ll be keeping an eye on this area to see if all the effort to standardize the internal engineering approach pays off!
December 30, 2024
Atlassian Taps AI to Automate Software Engineering Workflows

Atlassian this week added a generative artificial intelligence (AI) tool at its Team ’24 conference. Rovo uses knowledge graph technology to expose data to the Atlassian Intelligence platform for invoking multiple types of AI models.

In addition, the company revealed it has acquired Optic, a provider of an open source tool for documenting and managing application programming interfaces (APIs). Optic will be integrated into Compass, an internal developer portal (IDP) that Atlassian launched last year. Optic uses the OpenAPI specification to automatically track API changes as they occur across a continuous integration (CI) pipeline. It will now find OpenAPI documentation in code and publish it to Compass, which will then notify the rest of the software engineering team.

Finally, Atlassian also revealed it is now melding its project management application Jira with Jira Work Management to better streamline workflows. That next iteration of Jira, widely used to manage software development projects, will similarly be infused with AI capabilities enabled by Atlassian Intelligence, including Rovo agents to automate tasks.

Atlassian CEO Michael Cannon-Brookes explained to conference attendees that Rovo analyzes documents to provide a semantic understanding of documents. That enables the software to orchestrate workflows across both the Atlassian portfolio and third-party platforms connected via APIs. Organizations will also be able to use discovery tools accessed via a chat tool to find content, before they invoke agents to execute a series of tasks, said Cannon-Brookes.

All that will help DevOps teams launch a natural language query to, for example, surface a release plan, including source documents, for any application development project, Cannon-Brookes said.

In total, Atlassian is adding more than 30 agents to its portfolio that, to varying degrees, use AI to automate tasks across applications running in the cloud. Organizations can also opt to build their own agents using tools provided by Atlassian. Each agent will have specific types of objectives and goals assigned to it, noted Cannon-Brookes.

Ultimately, the future of collaboration will be defined by human and AI teammates working together to complete tasks, he added. Organizations will even be able to assign personality attributes to each of the agents based on user preferences, added Cannon-Brookes.

It’s not clear to what degree organizations will rely on AI assistants provided by specific providers of applications versus standardizing on a narrow set of agents provided by providers of, for example, cloud services running multiple applications. Regardless of approach, the level of task automation enabled by AI agents that take advantage of the reasoning engines embedded within LLM is about to considerably increase.

Naturally, each organization will need to determine how best to verify the tasks that the tasks assigned to an AI agent have been completed so there will be sone governance issue to address. In the meantime, however, DevOps teams should be compiling a list of tasks that might soon be better handled by an AI agent than by a DevOps engineer who should be resolving more complex problems.

May 2, 2024
Massdriver Adds Cloud Cost Controls to Internal Developer Platform

Massdriver has made available tools for tracking cloud costs and generating a cloud infrastructure bill of materials (Cloud IBOM) to internal developer platforms (IDPs) as part of an ongoing effort to streamline platform engineering workflows.

The Cloud IBOM makes it possible, with a few clicks, to map hybrid cloud resources such as databases, machine learning pipelines and serverless computing frameworks.

Massdriver CEO Cory O’Daniel said collectively, these tools reduce expenses by preventing misconfigurations that result in unexpected costs. Rather than solely providing a summary of costs incurred, the tools provided by Massdriver surface how much cloud services are going to cost at the time the developer is invoking them, he noted.

In the absence of that context, developers will nearly always overprovision cloud resources to ensure maximum availability regardless of cost, says O’Daniel. The overall goal is to help organizations reduce the total cost of IT by providing developers with insights into costs in real-time, he added.

Earlier this year, the company launched an IDP that comes with platform engineering tools that, for example, make it possible for DevOps teams to build a catalog of infrastructure components that developers can access within the context of guardrails they define. The platform also includes observability and visualization tools that make it simpler to track metrics and monitor the usage of cloud resources.

Massdriver is making a case for an IDP that provides built-in tools and workflows that enable organizations to embrace platform engineering as a methodology for managing DevOps workflows at scale. The goal is to reduce the level of expertise required to build and deploy applications in cloud computing environments, said O’Daniel. One of the primary reasons there are not more applications running in the cloud today is that these environments are still too complex to successfully manage, he added. An IDP abstracts away that complexity within a set of workflows that have been vetted by a DevOps team, he added.

There are, of course, multiple approaches to building an IDP, but rival approaches require too much effort to set up and maintain, said O’Daniel. A proprietary platform provides a more turnkey approach that doesn’t require DevOps teams to either build their own or customize an open source platform, he added.

It’s not clear at what pace organizations are embracing platform engineering, but there is a clear need to strike a balance between delivering IT services managed by a centralized team and the desire a developer might have to embrace new tooling whenever they see fit. Today, too much responsibility for provisioning IT environments has been pushed left toward developers who now have less time than ever to write code.

There is, naturally, no guarantee that reducing the cognitive load for developers will increase productivity. Building applications requires streaks of inspiration, but, in theory, an IDP should provide more time for inspiration to strike. DevOps teams, as part of that effort, should be continuously working toward eliminating any bottlenecks that have emerged in their DevOps workflows.

December 18, 2023
Atlassian Makes Compass IDP Generally Available

Atlassian this week made generally available a Compass developer experience platform through which DevOps teams can employ a software-as-a-service (SaaS) platform to centralize the management of the environments used to create code.

Taylor Pechacek, head of product for Compass at Atlassian, said Compass provides an instance of an internal developer platform (IDP) that is simpler for DevOps teams to set up and maintain.

At its core, Compass provides access to a software component catalog to track, search and discover the component and approved infrastructure that can be used to build and deploy applications.

In addition, templates enable developers to self-service environment setup using a set of policies that provide guardrails to ensure developers follow best practices for provisioning cloud resources and setting up pipelines.

Finally, Atlassian is making available scorecards based on the DevOps Research and Assessment (DORA) metrics defined by Google.

Compass is designed to be integrated with a wide range of continuous integration/continuous delivery (CI/CD) platforms and third-party observability, testing, collaboration and source code management tools, said Pechacek. The goal is to minimize context switching as much as possible for developers accessing the Compass platform, he added.

That approach also provides a level of consistency that both fosters collaboration and makes it simpler to onboard new members to a development team, noted Pechacek.

While IDPs as a concept have been around for some time, they are now central to any effort to embrace platform engineering methodologies to implement best DevOps practices at scale. The issue platform engineering teams need to come to terms with is determining whether they want to build and maintain an IDP versus invoking one provided as a SaaS application.

Regardless of approach, platform engineering teams are trying to strike a delicate balance between centralizing the management of development environments and the ability of developers to innovate by adding new tools to their development environment. Organizations clearly want developers to become more productive by spending less time managing development environments, but many of those same developers tend to jealously guard their prerogatives when it comes to the tools they prefer.

The challenge, of course, is that spending less time managing a development environment doesn’t necessarily mean more code will be written. All it does is afford developers the opportunity to write more code. The issue is that the writing of quality code only really occurs when developers are inspired, but at the very least, configuring a development environment shouldn’t stand in the way of creativity.

It’s not clear how many organizations are embracing platform engineering methodologies, but it’s apparent that more supervision is being brought to how applications are built and deployed within the context of a set of DevOps best practices. The challenge and the opportunity now is to make sure developers buy into, rather than resist, those efforts.

October 18, 2023
Tips For Managing Internal Developer Platforms (IDPs)

Platform engineering has emerged as a discipline to centralize configurations and standardize the DevOps process. As part of this practice, internal developer platforms (IDPs) are a key ingredient to establishing a paved road or “golden path” for developer users. But how do we ensure these IDPs are designed and rolled out correctly?

Well, the IDP concept is still relatively new, but some best practices are beginning to emerge. For example, treating the platform as a product helps encourage proper feedback. Reference architectures can guide initial designs. And testing first iterations with a single team can encourage buy-in around a proof of concept.

I recently chatted with internal developer platform experts, including Luca Galante, product and growth, Humanitec, and Markus Eisele, senior principal product marketing manager, Red Hat, to gather insights on how to best manage IDPs. Below, we’ll consider how to create golden paths that help optimize developer productivity and happiness and consider how these techniques affect the bottom line.

Start With a Reference Architecture

First, it’s a good idea to begin with a reference architecture when creating an IDP. This can help frame the initial infrastructure and its constituent components. For example, McKinsey recently debuted a sample IDP architecture, which has been reinterpreted for Azure and GCP. Platform engineering also maintains a tooling landscape that lists tool options for the various planes.

It’s good to note that an IDP is not a single product, like a platform-as-a-service. “An IDP is not a turnkey solution,” said Galante. “You can’t buy an IDP.” Therefore, there is not a one-size-fits-all enterprise-grade option that covers the entire stack. An IDP will most likely be a combination of open source and proprietary components.

According to Eisele, these new reference architectures are a good step in the right direction. “These IDP reference architectures are developer-centered views about modern application development platforms,” he said. “They shift the perspective from a sheer infrastructure perspective into what matters most for the developer.” And while a one-size-fits-all platform may not exist, the tools to build them do, such as Spotify’s Backstage or Red Hat’s Developer Hub, a platform for building developer portals.

Treat the Platform as a Product

Treating the platform as a product is another commonly shared best practice in platform engineering circles. “This is something that will make or break an IDP initiative,” said Galante. According to Galante, this platform shouldn’t be architected and then taught to developers in a top-down fashion—this can result in lengthy onboarding, broken functionality and ShadowOps.

Instead, successful platform teams take an existing team’s work and restructure it around a product manager to evangelize, said Galante. Another option is to keep your infrastructure team and develop a separate product team to start from scratch. Regardless of the exact organizational makeup, the product mindset is crucial for kickstarting and maintaining IDPs.

Have a Tight Feedback Loop

Part and parcel of retaining a product mindset is conducting research and having a tight feedback loop with stakeholders. For IDPs, this means understanding what developers need and not building on assumptions alone. According to Galante, all stakeholders should be given a voice, including DevOps teams, developer senior backend developers, junior front-end developers and even executives. Listen to these groups to understand their needs to frame the IDP correctly and sell them on the value of a platform. This is where the skillset of a PM is crucial to navigate these kinds of dynamics, he added.

“IDPs are about enabling developers,” said Eisele. While these tools should make them more efficient, they should also help them love their job more despite the cognitive load, he said.

First Attempt With a Lighthouse Team

Before rolling out a standardized internal development platform company-wide, it’s good to validate it in practice with a smaller group. Galante calls this a “lighthouse team.” Ideally, this team is more advanced with their deployment pipelines and is well-versed with cloud-native technologies, he said. This should help establish the right golden path with minimum friction and stress test it under real-world scenarios. After an IDP is validated in practice, rolling it out across other teams will be much easier.

Centralize the Platform Team Itself

Another facet to consider is the organizational makeup. Who will architect and support the IDP? Galante believes the platform team should be centralized instead of having various platform teams alongside each department or division, and such a team should consult various stakeholders throughout the application lifecycle. He also references Team Toplogies as a helpful resource for implementing platform teams.

At the same time, Galante admitted that at scale, large enterprises might spin up multiple platform teams simultaneously. Sometimes, this is done intentionally and could even bring positive competition to improve operations, he said.

Plan For Flexibility

Lastly, the big takeaway is that IDPs are not a silver bullet, and a reference architecture that works great for small agile teams might not transition well into a complex enterprise with much brownfield development. Furthermore, you run the risk of over-specificity, too, which could step on people’s toes.

Therefore, the platform must be flexible enough to adapt to new situations. Galante recommends first looking at your configuration management to see where the most time is wasted and ensuring you standardize the most common configurations.

Setting Golden Paths For IDP Success

Developer productivity practices have emerged as a response to exploding complexity in the last ten years, said Eisele. The old days are gone, and a brand new execution stack is required to architect and deploy applications. “All this led to a point where developers just want to get back to coding,” he said.

The golden path, or paved road as Netflix calls it, is like an approved path for traveling through the software development lifecycle. As Eisele puts it, it’s a form of best practices and a new way to replicate development knowledge across an organization.

“IDPs seem to be natural partners to cloud/standardized developer environments (CDE/SDEs),” said Ivan Burazin, CEO and co-founder at Daytona. He pointed out how Uber’s developer satisfaction increased dramatically upon adopting a platform called DevPod, an integral part of their IDP. He also cites successful CDE implementations at AirBnb as further proof of the power of internal developer platforms.

But to create these golden paths, IDP architects have their work cut out for them. They must juggle the input from multiple stakeholders and simultaneously balance complexity and abstraction effectively. The platform itself must be specific enough to reduce developer cognitive load yet generic enough to be flexible for outliers.

Thankfully, the aforementioned tools and best practices should help efforts toward designing golden paths for your internal developers. The next step will be testing the platform and measuring the success of these initiatives.

October 5, 2023
Internal Developer Platform (IDP) Reference Architectures
If you’ve been following the DevOps space recently, you’ve probably noticed the excitement brewing around platform engineering. Despite uncertainty in other areas of tech, it seems like platform engineering is poised for growth—Gartner even expects that by 2026, 80% of software engineering organizations will have established platform teams.

The platform engineering idea hinges on an internal tooling system known as an internal developer platform (IDP), which platform teams own and operate and is exposed to developers to improve their efficiency, satisfaction and release fluidity. IDPs have been the talk of the town. But what does an IDP actually look like? What sort of components are they usually composed of?

Well, at PlatformCon 2023, McKinsey representatives shared a reference architecture for building out IDPs. This framework was modeled after hundreds of internal platforms and could be a helpful benchmark for other organizations as they begin developing and standardizing their internal developer platforms.

Below, we’ll examine the sample IDP reference architecture and its components. We’ll also have a refresher on the platform engineering concept and offer some expert tips on overcoming potential obstacles while implementing platform engineering and IDPs.

Why Engineer Internal Platforms, Why Now?

First, for those new to the idea, let’s explain what platform engineering is. I’ve previously defined platform engineering as “maintaining an internal self-service platform that serves the needs of developer consumers.” The practice doesn’t necessarily supplant DevOps, rather, the experts I’ve spoken to view it as the next evolution.

This trend is essentially a response to developers overseeing more and more aspects of software life cycle management. Complexity in enterprise-level software delivery workflows has escalated in the last decade, leading to what The State of Platform Engineering Report describes as “increased cognitive load on engineers.” This is negative, as it could lead to shadow operations and burnout.

The fact is, most developers require the same sort of technologies, which could be standardized across the board. For instance, take the consolidation around containers and Kubernetes, infrastructure-as-code and the need to implement similar build configurations and security policies across applications.

By outsourcing the maintenance of tools for shipping and running cloud-native applications to a common platform, you take the burden off developers and free them up to code. Also, by centralizing DevOps tooling, you can enforce more governance and build out internal standards. Which should help reusability and overall deployment agility … at least, that’s the promise of it all. 😉

Sample IDP Reference Architectures

So, let’s turn to some real-world examples to see how this is being implemented. Mike Gatto, senior DevOps engineer, McKinsey, shared an internal platform reference architecture for AWS-specific environments:

McKinsey’s Internal Developer Platform reference architecture

As you can see, the architecture is divided into these subgroups. At a high level, they perform the following functions:
- Developer control plane: This is the version control, where developers publish their code. They provide a workload specification (in McKinsey’s case, a Score file) that codifies specific configurations.
- Integration and delivery: This includes the CI pipeline to build application code into a container and publish it to a registry. This notifies the platform orchestrator to initiate a build. Then, resources are provisioned. This area also includes a CD pipeline and infrastructure control plane.
- Monitoring and logging: This plane includes tools for observability, analytics, and application performance monitoring.
- Security plane: This area includes tools for secrets management, policies, and security vetting.
- Resource plane: This plane encompasses cluster management, database tools, networking, and messaging architectures.
“By separating developer and platform team concerns and allowing them to control the respective areas they’re experienced in separately, without requiring too much overlap,” said Gatto, “we’re making deployments more compliant and better aligned with client initiatives.”

Humanitec has also created internal developer platform reference architectures for GCP and Azure, displayed below.

IDP Reference Architecture for GCP

IDP Reference Architecture for Azure

PlatformEngineering.org has also expanded McKinsey’s original IDP reference architecture with a tooling landscape that adopts the same umbrella subgroupings but includes other pluggable tools.

The Internal Developer Platform tooling landscape

Roadblocks to Developer Platforms

Starting from a blueprint could be a big help to rapidly construct IDPs. Yet, there will still be some potential roadblocks associated with adopting an internal developer experience platform. One of them is creating common platforms that actually cater well to all environments.

“You can not and should not try to please everybody,” said Kaspar von Grünberg, CEO and co-founder of Humanitech. “Platform engineering is an 80/20 game.”

“Platform organizations often face a lot of pressure when it comes to meeting demands for building more services that are both agile and agnostic to cloud and data center environments,” said Venkat Ramakrishnan, VP, product management and engineering, Portworx by Pure Storage. “At the same time, almost all platform engineering teams are understaffed while facing the need to support large application development initiatives and developer teams twice their size.”

Tips to Get Platform Engineering On Track

Reference architectures for various clouds could help cater IDPs to different scenarios. But outside of that, it will take a product mindset approach to strategically accept feedback and iterate the platform. This will involve doubling down on developer experience and adopting cloud-native foundations that can scale.

“Platform engineering teams must consider adopting Kubernetes as it enables them to manage and deploy applications in production at scale, without having to manage applications at the individual level,” added Ramakrishnan. “Additionally, it may not be realistic to expand platform engineering teams alongside growing platforms, so teams should also consider leveraging automation and self-service capabilities to help them better manage projects at scale.”

Is ‘You Build It, You Run It’ Dead?

The idea of ‘you build it, you run it’ is starting to look a little antiquated next to the new and shiny platform model. It’s tough to say, but I’ll be keeping an eye on this area to see if all the effort to standardize the internal engineering approach pays off!
September 5, 2023
Product Mindset Key to Platform Engineering Success

Platform engineering has taken the world by storm, and for good reason. Good platforms reduce TicketOps, standardize config setups, and cut lead time and time to market. Despite the discipline’s new popularity, there remain some questions about what sets platform engineering apart from DevOps. The answer to those questions lies in the platform as a product approach–the heart of every successful platform engineering effort.

What is Platform Engineering?

According to Luca Galante, platform engineering is “the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era.” Platform engineers build an internal developer platform, which “consists of many different tech and tools, glued together in a way that lowers the cognitive load on developers without abstracting away context and underlying technologies.”

What is the Platform as a Product Approach?

A product approach requires user research, a defined mission, leveraging lighthouse teams, getting regular feedback, iterating and marketing your platform for internal buy-in. Together, these product management best practices will ensure your platform enables true developer self-service and finds the right level of abstraction.

“Build it, and they will come” is not a viable strategy for ensuring developers will adopt your platform. Similarly, forcing developers to use a platform is likely to fail. Mandated usage creates resentment, reduces trust and results in shadow ops. To avoid your platform’s untimely demise, you have to treat your platform like a product and sell it to your customers: Your developers.

This article provides a high-level overview of what a product mindset looks like in practice and why each component contributes to the success of your platform.

Do Your Research

Unless you gain a comprehensive understanding of developers’ pain points and what they’re already doing to mitigate those issues, you won’t build a platform people actually want to use. Hashicorp’s Michael Galloway shared this example of a user survey he conducted with his team while at Doma. The idea is to ask questions about the full software delivery process, from the planning stages of the interviewee’s work to debugging and testing. This way, the platform team can prioritize creating features to solve developers’ most frustrating problems.

Oftentimes, platform teams will find that teams have conflicting wants and perspectives on what the platform should be. A platform product manager will help translate disparate responses into an actionable and successful plan. Successful platform teams also research open source and commercial tools and integrate pre-built solutions into their platform where beneficial. They study blueprints, like the internal developer platform reference architectures developed by McKinsey, as a starting point.

Define Your Mission

Your mission statement gives your platform team a clear identity and priorities. It’s also a starting point for understanding and communicating your impact on the business to relevant stakeholders. Good mission statements are emotional and inspiring, simple but meaningful, match the longevity of the organization and are informed by user research.

Leveraging Lighthouse Teams

When starting their platform engineering journey, many organizations don’t know what to build first. The confusion arises from having two important yet conflicting priorities. On the one hand, your platform team doesn’t want to bite off more than it can chew by building too many features at once. On the other hand, platforms that are thin enough to cover the entire enterprise often fail to provide enough value to individual teams.

The way to strike the right balance is by leveraging lighthouse teams. A lighthouse team is the first group of people you’ll choose to build your organization’s platform for, so spend a lot of time with them to get it right. Cultivate ambassadors and evangelists and give them time to get the rest of the team (and the organization!) on board with your platform. Once you’ve built a platform that provides real value to your lighthouse team, repeat this process with more teams across the organization.

The lighthouse team approach is valuable because it helps platform teams avoid boiling the ocean while also ensuring your platform provides real value every step of the way.

Stay in the Loop

Initial user research is not enough to sustain a successful platform engineering effort. Platform teams should solicit regular feedback from developers and use it to iterate the platform.

Market Internally to Get Buy-In

Conversations about platform engineering often focus on developers, but developers are not your platform’s only stakeholder group. Your platform team will also need buy-in from managers, SysAdmins, DevOps engineers and the C-suite.

In his PlatformCon 2023 talk “How to communicate the business value of platform engineering,” Gartner’s Manjunath Bhat shared Gartner’s Enterprise Value Equation as a foundation for platform teams to understand and communicate the value of their Internal Developer Platform to different stakeholder groups.

The process includes assessing stakeholder priorities and concerns, identifying and defining value enablers, building a value map to map value enables to stakeholder impact, supporting the value story through outcome metrics, and communicating the “why” and realized value to the organization.

Sounds Great. What’s the Catch?

Applying a platform as a product approach can be difficult in practice. For example, many organizations struggle to afford the internal developer platform the same considerations as they would an external product. When working with internal developers, it’s easy for platform teams to make assumptions about what those users know, want and need. However, platform engineering requires avoiding these assumptions and always seeking user perspectives.

Another common pitfall organizations encounter is mandating platform adoption. Making the platform mandatory closes off the critical feedback loops platform teams need to gauge the success of their platform and make continuous improvements.

Conclusion

The platform as a product approach is what sets platform engineering apart from DevOps. It’s deceptively simple but important if you want to build an internal developer platform people actually want to use.

August 31, 2023
How Platform Engineering Makes Software Sustainable
The world is constantly becoming more and more aware of the importance of sustainability in every aspect of our daily lives from buying groceries to traveling, and the IT world is no exception. Indeed, sustainability is already gaining importance in software development. For example, in the latest version of Thoughtworks’ Technology Radar, “carbon efficiency as an architectural characteristic” is one of the new techniques of interest. Moreover, in its research “Predicts 2023: Environmental Sustainability Is Now an IT Sourcing Imperative,” Gartner states that “by 2026, 75% of organizations will increase business with IT vendors that have demonstrable sustainability goals and timelines, and will seek to replace vendors that do not.”

This shows that there is a growing need for sustainability. How is the software development industry responding to this need? Many great initiatives are arising, but platform engineering, in particular, helps make software more sustainable.

What is Platform Engineering?

According to Gartner’s definition, platform engineering is “the discipline of building and operating self-service internal developer platforms (IDPs) for software delivery and life cycle management.” This trend has gained momentum in the last few years, and an increasing number of organizations are adopting internal developer platforms (IDPs) to scale their development process. IDPs allow developers to self-service the technology they need, whether it is infrastructure, runtime resources or ready-to-use services. This helps speed up the development process and ensures higher-quality software.

Platform engineering is becoming more and more popular and foundational. According to Gartner, “by 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery.” Therefore, given the expected widespread adoption of IDPs in the next few years, sustainability emerges as a requirement. Ignoring or underestimating the importance of sustainability may cause organizations to waste all the benefits ensured by an internal developer platform.

Platform Engineering for Economically Sustainable Software

To survive and operate properly in the market, a company has to be economically sustainable and therefore stable. Usually, software is an asset that contributes to revenue generation for the company, but it still has maintenance costs. To make software economically sustainable, the total cost of ownership (TCO) needs to be lower than the generated income.

An internal developer platform can lower software product costs, making them more sustainable. How so? In a nutshell, by sharing resources that can serve multiple products and can be reused across multiple projects. Such resources include, but are not limited to, capabilities, processes, infrastructure, configuration, etc. Since the platform at scale manages shared capabilities, there product teams won’t waste time and effort replicating it every time. While lowering the TCO, this characteristic of platforms also contributes to reducing the software’s time to market.

Moreover, platform costs are shared across the whole organization and are not part of the TCO of each software product. This helps software to be more sustainable from an economic point of view.

Platform Engineering for Environmentally Sustainable Software

When talking about sustainability, most people usually think about the environment. While respecting the environment is one of the most important factors, it is important to note that sustainability is not limited to this aspect.

An internal developer platform can help software be more sustainable on two levels.
- Observability: The platform should collect and easily expose metrics on the environmental impact of the software. Such metrics can include hardware resource usage, energy consumption and CO2 footprint.
- Optimization: The platform should foster the adoption of sustainable patterns, like the ones provided by the Green Software Foundation. It should also include solutions that actively help reduce energy consumption, such as kube-green.
It is important that both of these capabilities are present and implemented in the platform. Observability without optimization is useless, and optimization without observability cannot be done.

Platform Engineering for Socially Sustainable Software

When speaking about sustainability, social responsibility cannot be forgotten. The first social responsibility of each organization is toward its members and employees. Does developer self-service also have social implications? Yes, it does; mainly by improving the developer experience.

Indeed, an internal developer platform helps organizations’ members to:
- Reduce friction between teams: Since everyone can have everything they need right when they need it, there will no longer be bottlenecks and misunderstandings between different teams. This contributes to a better working environment where everyone feels welcome and more productive.
- Decrease repetitive tasks and cognitive load for developers: Since the platform will manage and automate interactions with infrastructure and runtime, developers will no longer need to waste time and energy on such out-of-scope secondary tasks. This way, developers are more focused on their actual job of developing new features and they will feel more satisfied.
- Enable work from anywhere at anytime: Since the platform can provide all technology and capabilities whenever a developer needs them, it will no longer be required that people work together in the same location and at the same time. In other words, IDPs enable remote and asynchronous ways of working. This allows organizations to offer remote working and to create distributed teams, ensuring a better work-life balance for the team members.
All of this brings happier employees that will be more productive and satisfied with their work, which will reduce turnover.

Conclusion

The need for a more sustainable world is gaining momentum and involves every industry, including software development. No wonder Gartner included sustainable technology in the “Top 10 Strategic Technology Trends for 2023,” along with platform engineering. As platform engineering is being adopted by more and more organizations to accelerate software development, sustainability has to be addressed.

An internal developer platform can help developers build more sustainable software. Platform engineering can help in all of these aspects, improving not only the developer experience but also the organization and the environment. This is just the beginning of the sustainable path platforms should follow, but the outlook is very encouraging.
April 11, 2023