Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: All of Microsoft’s cloud services go down, everywhere except China. Redmond’s IaaS, PaaS and SaaS—including GitHub—were dead for several hours, and are still running unreliably—despite Microsoft saying it’s fixed the problem. (more…)
The speed of software development velocity is increasing, and to enable this, platform engineering is emerging to evolve the DevOps practice. Interest in platform engineering is rising—so much so that Puppet emphasized platform engineering in its 2023 State of DevOps Report.
A whopping 94% of respondents agreed that platform engineering is helping their organizations better realize the benefits of DevOps. The move toward platform engineering has increased over time, bringing benefits such as increased release velocity, reliability and efficiency.
Below, we’ll highlight some of the key takeaways from the report. We’ll define what platform engineering is and highlight its benefits. We’ll also consider obstacles on the path to embracing a platform-based development culture.
DevOps has been in use for a decade or more at most institutions, maturing steadily with new automation and infrastructure abstraction. The latest evolution of DevOps is a practice known as platform engineering—51% of organizations have already adopted this practice within the last three years and 93% said it’s a step in the right direction.
Platform engineering is the discipline of designing and building self-service capabilities to minimize cognitive load for developers and to enable fast-flow software delivery.
In essence, platform engineering is the process of creating standardized, self-service infrastructure that helps developers get up and running quickly. These self-service platforms typically handle capabilities like deployment (43%), operation of infrastructure (43%), monitoring production applications (42%) and security and compliance (42%).
These shared internal platforms are maintained by a central platform team that embraces a product mindset to iterate the technology based on user feedback.
Platform engineering is relatively new— 27% of platform teams have formed within the last two to three years. Notably, big tech companies hopped on the bandwagon years ago, and non-tech companies are now following. Regardless, 70% believed their timing was “just right.”
The goals of a platform engineering team are numerous. According to the Puppet report, 54% said problem-solving is the top goal. This is followed by educating and empowering developer teams (47%), setting best practices (46%) and optimizing iteration (44%). A platform engineering team’s scope of work is primarily focused on automating workflows and building and managing infrastructure.
The ‘correct’ number of internal platforms is a moving target. The majority of organizations (57%) have between two and four platforms, and 30% have five or more internal self-service platforms. This demonstrates that although platform engineering is somewhat centralized, it’s also flexible to the needs of an organization.
So, what are some benefits as opposed to traditional DevOps? Well, the most common benefit is improved system reliability, at 60%. This tracks, as a standardized platform could reduce human error and missteps in the development workflow. Following this benefit is improving efficiency (59%), speeding up delivery time (58%) and improving workflow and process standards (57%).
The most common driver of adoption is increasing the speed of delivery. Slight more than half of respondents (51%) said development speed has somewhat increased since the inception of the platform team, and 42% say it’s increased by “a great deal.” Other top drivers include the need to scale, free up engineering time, control infrastructure spending and avoid duplication across teams. Those with more mature adoption tend to be more satisfied with the result, indicating that results increased over time.
Of course, no new technological introduction comes without its hurdles, and platform engineering is not immune from certain pain points. For example, 34% said that cycle time is slower than expected. This might be due to nascent industry adoption and a lack of best practices.
Just under one-third (32%) also noticed resistance to platform team adoption, and 32% reported a lack of communication around changes. Organizations will likely have to increase transparency and reduce siloing to encourage productive platform adoption that satisfies the needs of various stakeholders.
According to the report, product management skills can help realize these goals. Some of the most fundamental product management skills here include strong communication skills (61%), problem-solving expertise (60%) and the ability to foster collaboration (54%). Many companies will also have to convince senior management to invest in platform engineering—nearly half agreed that senior management doesn’t yet understand its value.
Confidence around platform engineering is increasing, especially for teams with mature DevOps approaches. As a result, the practice will likely gain more traction in the coming year—71% of respondents reported their firm plans to hire people with platform engineering experience over the next 12 months.
Over the next year, organizations that adopt the practice will want to increase awareness of platform capabilities and set expectations across the organization. This will also include setting governance policies and satisfying the needs of specific developers, departments and the organization as a whole.
Above, we’ve scratched the surface of the current state of platform engineering, its benefits and its pain points. For plenty of other insights, you can pick up a copy of The State of DevOps Report 2023: Platform Engineering Edition from Puppet by Perforce here.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The FAA’s NOTAM database gets corrupted, and Threema shows why DIY cryptography is bad. (more…)
Although a relatively new concept, site reliability engineers (SREs) have become crucial for DevOps teams, helping to solve an array of operational problems such as network availability and user experience. However, in previous years, some people have questioned the longevity of such a role.
This article will discuss several reasons why site reliability engineering roles are here to stay and why they have become critical to DevOps.
A site reliability engineer helps connect development teams and IT operations by completing tasks previously assigned to operational roles. SREs use a range of automation tools for troubleshooting purposes, creating software systems that are both scalable and reliable.
The role revolves around automation and standardization, as cloud systems become increasingly prevalent. SREs also play a vital role in ensuring the user experience of systems is of the highest standard.
Let’s take a look at two of the primary reasons SRE roles are here to stay.
The key reason for the rise in popularity and the expected longevity of site reliability engineering roles is user experience. In the modern climate, user expectations are very high and more dedicated roles are required to help deliver a service that is not only acceptable but attempts to exceed user expectations.
Even back in the 2010s, users were much more patient in terms of website load speeds and how quickly they could access information. Even crashes and load failures were tolerated without the user exiting a website.
However, that has all changed due to the competitiveness of the market, as well as the growth of remote working, which has increased dependency on online resources. This means that if a service is unavailable for any period, users are unable to do their job.
An SRE’s role includes optimizing systems, ensuring they perform at the required level and that resources are always available to those that need them. This makes SREs critical to DevOps teams that may not have the capacity to focus on meeting user expectations sufficiently.
Another factor that has added a lot of value to the role of an SRE is their ability to provide much-needed support for software environments and architectures that continue to develop in terms of complexity. Kubernetes is one such architecture that requires dedicated attention.
Applications that are distributed across systems such as Kubernetes and other cloud-native platforms require specialized personnel that have the necessary knowledge and experience. Machine learning is also becoming more prevalent and requires special focus from a performance perspective. In 2020, just more than 22% of companies had machine learning models in production for one to two years.
This level of complexity is only expected to grow further over the coming years as cloud-native systems become the norm. This is why new roles must be created that go beyond that of a normal IT engineer.
Like any IT role, the role of a site reliability engineer always faces challenges that have led people to question whether SREs are a long-term solution or whether it is a niche specialization.
The role of an SRE was created by hyperscale companies like Google that manage extremely large IT systems. For companies that do not have such a wide and varied IT infrastructure on a global scale, there are questions about whether such a role is necessary.
However, as previously mentioned, the growth of cloud-native technology shows no sign of slowing and, with this growth, opportunities for SREs at smaller organizations may arise.
The ambiguity of an SRE role can be considered a downfall in some cases, with many businesses unsure of what these professionals offer when compared to a traditional systems engineer. In many cases, SREs are deemed to be a hybrid of a software engineer and an IT operations role.
Fortunately, DevOps has realized the potential of SREs and their ability to provide an important bridge between developers and engineers, helping to achieve dedicated tasks to a high standard.
Although both roles are connected and share some similarities, there are some clear differences.
As you can see, the two roles differ significantly but SREs can play a critical role in ensuring the DevOps team delivers a product that offers maximum performance.
The core benefits of employing professionals in a site reliability engineer capacity are high levels of product performance and reliability to meet user expectations while helping to manage complex IT architecture.
In the future, organizations that manage a large amount of cloud-based IT assets may struggle to deliver efficient products without such professionals. As such, demand for SRE roles is likely to grow.
Over 1.3 billion people live with some form of disability. This accounts for almost 20% of today’s workforce in the U.S. While great strides have been made in closing the disability inclusion gap in the workforce, more needs to be done.
As the popularity of hybrid and remote work continues to grow, workers who live with a disability and work from home increasingly need accessible collaboration tools made available to them.
To meet this demand, developers need to build with accessibility in mind. By doing so, they can create more accessible software to help workers with disabilities overcome any unique challenges they may find when interacting with technology while creating an enjoyable working environment for everyone.
There are many ways to do this, however, here are some best practices and key learnings to help developers build with an accessibility-first mindset.
Innovation is always about challenging the norm, and this is especially the case when it comes to developing more equitable products. There are many ways to do this, depending on what processes are already in place. Still, one area that I believe can always be adapted is the way accessibility is thought of and managed internally, especially with regard to product design.
While the vast majority of companies today prioritize accessibility in some way or another, often it’s done by having a team—or, in the case of smaller departments, an ambassador—tasked with managing this work. This approach often leads to accessibility being seen as an add-on to design, something that happens at the end of the process. In reality, it’s better for developers and DevOps teams to take a hub-and-spoke approach to accessibility. In this model, there is one central source of expertise that people can go to for advice, but individual teams also are tasked with working on and being mindful of accessibility at the same time. The results as a collective would be far more collaborative and impactful.
Likewise, challenging the status quo could be as simple as encouraging conversations about accessibility with your team. If accessibility is a subject you want to discuss with your team but don’t know how to bring it up, remember that most accessibility problems stem from a lack of awareness. It’s usually an oversight, and ideas begin to flow once the conversation starts. So why not kickstart the conversation? Come prepared with user stories and potential solutions and your team will immediately start seeing all the appropriate changes that need to be made.
When building any software, it’s important to gather feedback from all kinds of people so you can be sure you’re building the features and experiences people need.
When we begin a project or an update, we strive to understand how current functions or other similar products are (or are not) accessible so we can better understand our customers’ expectations or frustrations. There are a number of ways to do this, but perhaps the most broadly impactful is hiring people with disabilities and having an open discussion with them about the product. Along with hiring, it’s also valuable to contract specific companies that employ people with disabilities to test products and give their feedback.
Receiving this feedback is key to developing a successful product and usually results in a more flexible and well-rounded user experience. For example, a text-based app with extensive text scalability designed for people who are blind or low-vision will be equally as useful to a user who prefers their text to be small or uses the product in a language requiring different string lengths. Similarly, captions designed for people who are deaf or hard of hearing can be useful to anyone who might be in a noisy environment or who prefers to consume written rather than audio content. When you make a product work really well for a particular group, you inherently create a better and more adaptable experience for everyone; you are meeting users where they are.
It’s incredibly important when you’re having conversations with people with disabilities or when designing a new product to not make assumptions. For example, early in my career, I would have assumed that an image-based program would be completely uninteresting to someone who is blind; I now understand that is categorically not true. By integrating interesting and vivid descriptions of images into the product, an image-based program can become accessible and engaging to all.
By avoiding assumptions about what someone can or cannot do based on their abilities, we open ourselves up to a world of creative solutions and possibilities that we might have otherwise not considered.
While people with disabilities ultimately provide the best feedback, don’t forget to spend time on your own or as a team actively challenging your design and looking for ways to improve its accessibility. Spend an hour or two as a team using your product with a screen reader such as VoiceOver or TalkBack or use your application using only the keyboard but no mouse. By constantly engaging with and testing your own design, you will better understand the user experience and may catch bugs or accessibility issues on your own.
Twelve years ago when I first started to focus on prioritizing developing for accessibility, there was very little documentation on accessibility in this space–especially when designing for mobile technology. This caused a great deal of frustration and often slowed our work down as we tried (and failed) to get it right. Sometimes this involves a delicate balance of adhering to your product’s established standards (i.e. how things should be labeled or how things should look) while solving for accessibility and efficiency in tandem.
Since then, things have changed a lot in the developer space. Today, there is a lot more material out there about the topic as well as industry standards to abide by. Universities are now offering more accessibility training within computer science courses, content creators with disabilities share more of their experiences, and companies are more open about sharing accessibility resources.
However, trial and error is still vital to building with an accessibility-first mindset, especially as we continue to innovate in this sector. So be patient and trust the process.
Inclusive design results in better products for everyone and, in particular, for all the people around the world experiencing some form of disability. Make sure you are a part of the solution and share your learnings with your developer community to continue this movement.
As a developer, the more time you spend learning and listening to your community and making them your number-one resource the more understanding you will be of their needs and how you can create the ultimate user experience. In turn, this empowers users to do their best work regardless of their abilities or circumstances.
As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.
Site reliability engineering (SRE) teams and platform engineering teams share similar goals—like maximizing automation and reducing toil—and similar methodologies. But they have different priorities, and use somewhat different tools to achieve them.
What are SREs? What are platform engineers? How is each role similar and different? This article will answer all your questions.
The main responsibility of site reliability engineers is ensuring that IT systems are reliable, which means that the systems meet performance requirements. Part of their job includes working with developers and IT operations teams to maximize the reliability of applications as they move down the software delivery pipeline.
Meanwhile, platform engineers focus on managing and optimizing the software delivery process. You could think of this role as one that translates DevOps principles into practice by finding ways to implement continuous delivery, continuous improvement and other DevOps priorities.
Thus, both SREs and platform engineers play a role in managing the software delivery process and finding ways to make it as smooth and efficient as possible. They also share a common enemy—toil—in the sense that both groups seek to avoid manual, inefficient processes by automating as much as possible. SREs automate complex incident response operations using tools, for example, while platform engineers might deploy methods like GitOps to automate some aspects of software delivery.
That said, SRE and platform engineering are distinct roles. The key differences boil down to:
So, although SREs and platform engineers may use similar methodologies and possess similar skills, their day-to-day operations tend to look quite different.
Given the skills overlap between SRE and platform engineering, some engineers are a good fit for either role. But which one is better, from a career perspective?
Salary-wise, there’s not a whole lot of difference. Glassdoor pegs the average salary of platform engineers at about $115,000 compared to $127,000 for SREs. SREs may earn a little more (and it’s worth noting that SRE salaries can vary widely), but the difference in pay is not as great as that between, say, IT operations engineers (who make a mere $77,000, according to Glassdoor) and SREs.
The roles are also similar in terms of demand. Unlike software engineering or even DevOps engineering, SRE and platform engineering are both roles that have begun to become popular only in recent years. SRE as a concept has been around for a while—it originated at Google in the early 2000s—but it wasn’t until recently that most companies began seeking SREs for their teams. Likewise, platform engineering didn’t become a thing until the advent of microservices and cloud-native everything pushed companies to devote more engineering resources to manage their increasingly complex software delivery processes.
What this means is that, at least for now, it’s easy to find jobs in SRE and platform engineering as long as you have the right skills. Many companies are eager to hire for both types of roles.
Choosing the role that is right for you, then, depends mostly on your preferences and how you like to work. The life of a platform engineer is likely to be somewhat more predictable than that of an SRE because platform engineers don’t have to take the lead in managing incidents when something breaks at 2:00 a.m. On the other hand, SREs enjoy a more diverse set of responsibilities, which can make the job more exciting—especially if software delivery tools like Git or Jenkins don’t make you excited to get out of bed in the morning.
Platform engineers and SREs play central roles in automating complex processes and eliminating toil. But they tackle these issues from different angles and they use different tools. The right role for you depends on what you enjoy doing most.
For years, self-service has been touted as the salvation of DevOps. Why? Well, every department in your organization has its own reasons for putting self-service DevOps at the top of its wish list:
● Ops teams need to ensure high availability, performance and security
● Dev teams need speed, self-sufficiency and transparency
● Security teams need guardrails, compliance and accountability
● And, of course, the C-suite needs ROI, strategy and insight on top of everything else.
These goals often cause tension—especially between Dev and Ops. Verifying security takes time that developers usually don’t have. And once at scale, even the fastest manual processes break down. Like at the U.S. Department of Defense, for example, with a workforce of close to 200,000 internal and external developers.
Self-service DevOps platforms promise virtual self-service kiosks for software delivery, uniting teams and creating a frictionless path to independent developer access and provisioning. According to the State of DevOps report from Puppet, self-service DevOps is a hallmark of “highly evolved” DevOps organizations.
Many self-service solutions are available, but has anyone actually figured it out yet? After all, if everyone is happy with their self-service approach, why are they still searching for answers?
True self-service DevOps promises a wide range of benefits: It lets Ops stop rushing to put out fires and shift to a big-picture strategy; it makes compliance simpler with automation, safety guardrails and clear accountability. And, there is the promise of tremendous capital efficiency.
It’s no surprise that so many vendors are rushing into the space, offering so many flavors of approaches that Baskin-Robbins would be jealous:
● Internal developer portals
● Shared service platforms
● Workflow automation
● ChatOps/Chatbots
● Service catalogs
● API marketplaces
● Cloud competency centers
But let’s face it: In reality, most of these approaches fail. And when self-service isn’t working as it should and companies fall behind on their service request queue, many DevOps teams default to risky practices that aim to expedite operations but actually expose the company to attack:
● Over-privileged and over-permissioned accounts that give broad access to anyone who gets their foot in the door, as happened with the SolarWinds breach of 2020
● Inactive accounts that leave a door open to cybercriminals, as we saw with the Colonial Pipeline breach of 2021.
What you might not realize is that these two risks are so prevalent that IBM Security X-Force penetration testers managed to use them to hack into 99% of client cloud environments. So when we say self-service DevOps is failing, it’s failing big-time.
So what’s the solution? Why is true self-service DevOps so hard to achieve?
There are actually a number of reasons why today’s self-service DevOps platforms fail:
● They break end-user workflows, forcing users to change their behavior and learn new processes.
● They force devs to switch context which kills productivity.
● They need domain expertise to operate, reducing the target audience that can use them.
● They’re heavily reliant on human-in-the-loop involvement, often creating more work.
● They require significant effort to update and maintain workflows and meet or exceed SLAs.
Most importantly, with current self-service DevOps solutions, users are often flying blind. They don’t know what they don’t know; for instance, how to locate, search, identify, create and use resources and workflows.
Imagine, for example, choosing a self-service checkout to avoid a long line at the grocery store cashier. But then …
a) You get stuck behind someone frantically trying to scan a wrinkled, expired coupon.
b) You get stuck behind someone who doesn’t understand barcodes and scans all their items twice.
c) You forget that alcohol requires age verification, causing an alarm bell and flashing lights to start going off above the register (spoken from experience; embarrassing).
d) All of the above.
In all these scenarios, a human attendant must come running to fix things, defeating the whole point of self-serve. Self-service is only self-service if it’s easy to use even when things go wrong.
With most so-called self-service DevOps solutions, if something goes wrong, you’re back to waiting—drawing the operator back into the loop—just like with the old system.
To achieve true self-service DevOps, you need a few elements in place.
Headless UI
Most developers are (gasp!) human beings. So the best interface is no interface at all. Why not let developers express their needs freely in natural language using existing tools such as Slack, Microsoft Teams or a command line interface?
Workflows
Developers are frustrated when processes don’t make sense or take more steps than necessary. Let’s adopt workflows that actually handle all the nitty-gritty—so that all developers need to do is ask (in their own words) to be granted access to resources, information and workflows.
Intelligence
For developers, there’s nothing more frustrating than not knowing where to turn for the information you need. True self-service DevOps, powered by AI, will use your existing data to provide a single source of truth with all the answers and insight devs need.
End-User Feedback Loop
An ideal self-service DevOps platform will ask the developer for feedback that is then used, in turn, to improve the platform itself. First, because everybody enjoys having their voices heard and second because this reinforces the role of the platform, which is to serve DevOps by improving the end-user experience. Similar feedback loops have been used in apps like Waze, which uses social navigation to continuously improve the quality of its end-user experience. Self-service DevOps is next in line.
Once you’ve accomplished all of this, you can work more efficiently, providing immediate on demand assistance so your Dev and Ops teams can work without interruption.
How will you know when you’ve achieved true DevOps self-service? When you forget you ever did things any other way—and wouldn’t go back for a million dollars. When you wake up one day and realize that your platform not only lets you build trust and accountability but also gets all your teams working together…and—(gasp) dare I say it?—having fun.
That will be the last DevOps platform you ever buy.
A lesson I have learned over my 30 years in the tech world is that technology and frameworks never truly die—they evolve and they grow. When I read a recent article proclaiming, “DevOps is dead!” and that we should all be embracing platform engineering, I had a chuckle.
No, DevOps isn’t dead, and rumors of its demise have been greatly exaggerated. In fact, DevOps is more alive than ever and the truth is, it is going wherever software goes. When I went back and read the article again, I realized that it appeared on a website owned by a venture capital (VC) company and that the article itself was a sponsored post from a company promoting platform engineering. That’s a good reminder to take that article with a grain of salt. I also realized that the headline was meant to be controversial, if not downright clickbait.
That said, I feel that the article demands a rebuttal. To be fair to the author of that article, I believe the point they were trying to make is that there is a major issue with DevOps—we have put too much on the developers’ plates, and that one more straw may break the proverbial camel’s back. Certainly, this is a valid point. We need to balance what we ask our developers to do in relation to the rest of the team. We see this with security in DevSecOps; we see it with testing and we certainly see it with operations. In fact, in many ways, the rise of the SRE has better defined the ops role thereby lessening the load on developers.
I have no doubt that platform engineering will also lessen the load on developers and better define the role of a platform engineer. However, SRE did not kill DevOps and neither will platform engineering. At one time, it was very fashionable to talk about a full-stack developer. Today, many people think that the so-called full-stack developer is a myth and that role is not really what is needed in most organizations. The job of a developer, though, is never and will never be easy, even with the advent of AI/ML-generated code. The developer always has many plates spinning in the air at any given time. And to that point, the high levels of compensation they are paid is not unwarranted nor is it unearned.
I look forward to seeing how the platform engineer role and platform engineering as a practice evolve and how they interact with SREs and DevOps teams as well as developers, security testers, etc., but I am fairly certain that no matter how the role evolves, it will not be the death knell for DevOps.
As we proclaimed for our upcoming DevOps Experience event on November 16, 2022, it is going where no DevOps has gone before. We see DevOps-of-things; we see DevOps at the edge. We see distributed DevOps, too. DevOps has taken root in our software development life cycle and beyond. While the frameworks may change and evolve, DevOps changes and evolves with it.
If you say DevOps is dead, I disagree. I say long live DevOps!
If you’re creating a developer tool, you’re bound to run into this dilemma—you know the inner workings of it really well, but others don’t. And without solid documentation and samples to follow, it can be challenging for another person to get started. Friction with the tools we use can contribute to burnout, inhibit creativity and slow down innovation.
Developer experience (DX) is becoming integral for any software company to consider. I recently met with Christine Spang, CTO and co-founder of Nylas, to consider the benefits of investing in developer experience. Below, we’ll take a look at what exactly DX is and consider some best practices for implementing it.
Developer experience (DX) is similar to user experience but for developer-facing tools. DX considers the developer’s journey while developing with a certain framework, language or platform. “Developer experience is the amount of friction a developer experiences between having clarity on what they want to implement and getting to the point where you have created that thing,” said Spang.
For example, consider integrating with an API to solve a common use case like payments integration. There are many steps involved here, from reading the reference documentation to getting an API key, testing calls and understanding how the service responds. After developers initially integrate the API into their application, they may also have to make continual updates as the service versions. Throughout this journey, DX can be thought of as “how hard it is to figure out how to translate from idea to output,” said Spang.
It’s also important to note that DX isn’t only relegated to public-facing platforms. Having a quality developer experience is just as crucial for internal software. The urgency to improve internal developer experience is increasing with the surge of APIs being developed, especially since the majority of them (58%) are private APIs, according to Postman’s 2022 State of API Report. Depending on the size of a company, internal DX can encompass many areas.
So, what are some methods to improve developer experience across the board? Well, having up-to-date documentation, code samples, better examples and standardized documentation rank high as positive developer experience traits. Spang also shared some helpful best practices to improve DX:
Provide scaffolding for the most common cases. Measure patterns around the kind of things people are building with the platform and offer scaffolding to support the common use cases. This might equate to a getting started guide for a common scenario.
Offer sample code libraries. It’s a lot easier to edit code than write it from scratch. With that in mind, create pre-baked environments where developers can play around. Offering working apps, use case templates and glue code can be a practical means to help developers get going more seamlessly.
Start with the reference manual. Walkthroughs and sample code should never replace a complete reference manual. It’s essential to have up-to-date documentation to cater to advanced users from the get-go. From here, you can generate Postman Collections or interactive sandboxes built around the reference specification.
Measure DX success and iterate. One way to monitor developer experience metrics is to record the onboarding flow and measure conversion rates. Where APIs are concerned, this can be tracked by asking how many people get through to making calls? What is the mean time it takes to make the first call? Knowing such metrics can inform how effective your tool’s self-service features are.
One point that’s important for DX is walking the fine line between too much custom work and focusing on the core project. On that note, Spang recommended “making sure whatever you’re building maps to the overall vision of the company.” In essence, if you’re not a consulting shop but a product company, you should stay true to that objective.
“You want to make the path for what people commonly want to do very easy,” said Spang. “Yet, you don’t want advanced users to feel constrained by the platform.”
For those creating developer-facing services, it’s helpful to have a framework to model. So, what are some examples of quality developer experiences?
Well, top public APIs are great examples of where the bar is set for developer expectations these days, said Spang—services like Twilio or Stripe demonstrate relatively seamless developer journeys. Spang also pointed to companies with a strong DevOps and CI/CD background, like Etsy, who have for years invested in usable backend infrastructure that accelerates internal development.
For other sources of quality developer experience, consider the high-ranking developer-favorite technologies from the latest 2022 Stack Overflow survey. Engineers love working with technologies such as PostgreSQL, AWS, Phoneix and Docker. The newer, trending languages tend to be increasingly usable, too. For example, 86% of developers said they love working with Rust while 76.65% dread working with Objective-C.
An investment into developer experience can help improve agility and developer satisfaction, said Spang. For one, an easier path for development increases the velocity in how you ship and evolve projects over time. Secondly, developer experience equates to less burnout and less turnover, both of which can have real business impacts.
The software development industry is growing like wildfire, bringing in waves of entry-level developers each year. Interestingly, the average experience level is decreasing, increasing the need for more abstracted tools and toolkits, said Spang. In this environment, developer experience undoubtedly shines as a competitive advantage.
So, if you’re creating developer-facing tools, consider the experience others will have when using them. Increasing developer experience may just be a defining factor in the project’s success and the long-term happiness of your developer community.
One main advantage of the cloud is that it offers developers flexibility and a self-service solution. Instead of having to contact the central infrastructure team whenever a dev team needs a new server or a cluster, the dev team can do it themselves through a simple web UI. This change drastically improves productivity. But there’s a caveat: Most companies don’t actually want to use the default portals provided by cloud providers. There are a few reasons for this: Security, simplicity or the need for custom, company-specific options, to name a few. That’s why we now see a move toward internal developer portals. In this post, you’ll learn what internal developer portals are and how they can make your developers’ lives easier.
An internal developer portal is a self-service application and data store that lets developers and managers track and organize everything their engineering teams build and operate.
Imagine that you need to create a new microservice. Your developers will first need to provision infrastructure for it; for that, they’ll use a cloud provider portal. Next, they’ll need to create new CI/CD pipelines or modify existing ones. Then, they’ll probably need to implement some changes to existing components like the API gateway and database. Lastly, they’ll probably need to add some documentation or make their new microservice visible to the rest of the company.
Overall, they’ll need to touch a few different UIs or portals. Now, imagine they can do all of that from one place. That’s what we call an internal developer portal. It’s a self-service portal that lets developers catalog and understand relationships, ownership, resources, external service data and anything else they need to understand and operate their software.
Now that we know what an internal developer portal is, let’s take a step back and discuss why internal developer platforms (IDPs) are on the rise. The main reason is because of the increased complexity of both software and infrastructure. Especially when we’re talking about cloud-based microservices environments, it’s challenging even for senior engineers to keep track of all dependencies and things that must happen every time changes are needed. Microservices architectures are heavily decentralized. Although you’ll still see the typical three-tier software architecture, in most cases, it’s no longer easy to tell by looking at a random component where it fits in the bigger picture just by its name or server location.
The second reason is that it’s simply a natural next step in the evolutionary process for self-service portals. Old-school, on-premises-based environments barely had any self-service capabilities. Do you remember how difficult that was? To request a new server, you had to email the central infrastructure team. If you were lucky, you’d get what you needed in a matter of hours or days. But more often than not, it took a week or two. And if you made some mistake in the initial request, that would cost you an extra week (or more) of waiting. More advanced companies? Even when they created some custom self-service solutions, these were limited. Often, it wasn’t much more than an online form to be processed by the infrastructure team which often became a bottleneck for all dev teams.
When most companies started moving to the cloud, they began to see the benefits of self-service platforms. The ability to create servers ad hoc within a matter of minutes led to increased developer productivity, faster time-to-market and better quality software. But again, cloud-provider platforms are suitable for managing cloud infrastructure; you need more than that for a complete solution. Your developers can be more effective at creating and changing cloud infrastructure, but everything else was still done separately. And that’s how we naturally evolved into the demand for internal developer portals. To bring even more flexibility and clarity while increasing your dev teams’ productivity, you need to take the next step in the self-service platform evolution and combine all the things developers need into one internal developer portal.
Also, from a business perspective, internal developer portals bring clarity and offer a good overview of all your infrastructure and software development efforts. Even if you’re in the cloud and your dev teams can manage the infrastructure as they choose, you still need some control and oversight—for example, to avoid wasting money on test servers that developers forgot to delete. Sure, you can get simple reporting from your standard cloud-provider portal, but what if you’re using multiple cloud providers or have a hybrid-cloud environment? An internal developer portal can benefit both developers and their managers. Long story short: IDP portals are the future of software development.
How do you create an IDP for your company? You can build your own from scratch, use some OSS offerings or go for ready-to-use products.
The first option would give you the most flexibility and ability to perfectly tailor the solution to your needs and the needs of your developers. However, building your own IDP from scratch is almost like building your own operating system from scratch. It requires a lot of work which could negate the main benefits that IDP is supposed to bring. You want to implement IDP to increase developer productivity and decrease time-to-market, but it would take you months to build it and years to maintain it to keep it working properly with constantly changing cloud APIs.
The next option is open source IDPs. Since IDPs are, in general, quite complicated pieces of software, there aren’t many options to choose from. The two most popular are Backstage and Gimlet. Backstage is more of a DIY platform for building developer portals, which means you’ll still need to spend quite a lot of time creating something that actually benefits your company. Gimlet is quite popular, too, but it focuses on Kubernetes-based environments. Both are great tools—don’t get me wrong—but if you’re looking for something ready to use and feature-rich, consider the third option.
The last option is to use a ready-to-use product. Ready-to-use solutions combine the IDP functionality with incident response capabilities as well as metrics, dependencies visualization and filtering across multiple clouds.
Internal developer portals are gaining popularity for good reason. In fact, many companies have been trying to build IDPs for years without even knowing the solution had a name. This is because an IDP-like solution will naturally evolve in any company that cares about developer experience and moving faster. The problem is that it usually takes years to come up with a single solution that fulfills all their needs. Another problem is that most of these organically created internal developer portals focus only on the tech side and skip the business needs. That’s why a purpose-built tool can be much easier and faster to implement while also doing its job much better.