Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Rethinking return-to-office mandates and a ridiculous, ancient Windows bug. (more…)
Category: Platform Engineering
-
Most Organizations Benefit From a Mature Cloud Strategy
Today’s cloud adoptions are wide and varied. Large enterprises are composed of vast, hybrid multi-cloud strata, often incorporating legacy technology acquired at various times and haphazardly equipped from department to department. But a fractured cloud approach can leave IT wanting in terms of shared automation and standard security policies.
Not all organizations have addressed best practices around common areas like CI/CD, resource optimizations, or common policies for multiple clouds. And many teams are just getting the hang of the new cloud-native stack. But studies show that a mature approach to cloud operations can pay dividends in the long run—and this approach hinges on cloud platform teams.
HashiCorp recently released the findings from its third annual State of Cloud Strategy Survey, which surveyed 1,000 technology practitioners on their cloud adoptions. In short, the report discovered that increased cloud maturity equates to increased overall value. Below, I’ll distill the major findings from the report to better understand how cloud-native operations are maturing.
State of Cloud Maturity
Most organizations are moderately increasing their cloud investments. Overall, 56% increased their cloud spending in the last year. Increases were much higher for higher cloud-maturity organizations and stilted for lower-maturity organizations. The findings demonstrate general confidence in cloud adoption, even amid economic uncertainties.
There is also persistent optimism in multi-cloud strategies. In fact, 73% of companies said a multi-cloud strategy helped advance or achieve their business goals. And to successfully accomplish a multi-cloud strategy, respondents ranked security, uptime and availability, infrastructure scalability and regulatory compliance as top factors.
However, there is still a lot of waste in the cloud. Nearly all organizations (94%) said they have avoidable cloud spending. These expenses most commonly stem from areas like over-provisioning resources, idle or unused resources, cost premiums or a lack of expiration date on cloud resources.
Benefits of Maturing Cloud-Native Operations
Overall, there are many reasons for adopting a multi-cloud approach—the report found reliability, cost reduction, security and governance and digital transformation as top motivators. And on average, high-maturity cloud adopters tend to reap the rewards more often, the report found. Some of these top rewards ranged from a stronger security posture to better visibility and increased automation. Furthermore, half of high cloud-maturity organizations reported successfully saving money using multi-cloud.
Most interestingly, perhaps, is that high-maturity cloud adopters said they are better equipped to attract, motivate and retain talent than lower-maturity cloud adopters (74% compared with 48%). It’s generally easier to pull in talent when operating with attractive cutting-edge technologies that look good on a resume.
Spotlight on Platform Teams
Even then, skills shortages remained the top bottleneck in most organizations’ ability to operationalize multi-cloud and set up platform teams. This can be side-stepped by reskilling or upskilling staff, incorporating more consultants or increasing the budget. But the best strategy is to standardize on a common operating model.
In that vein, investments into the platform engineering concept are rising. In fact, 92% of organizations are adopting, standardizing or scaling platform teams. Low cloud maturity firms tend to lag on platformification, most likely due to lacking the necessary skills to build and maintain internal developer-facing infrastructure, like the tools used to measure and improve developer productivity and satisfaction at LinkedIn. Others are still on the cusp of exploring how to implement it.
The idea of platform engineering is still taking root, so it was interesting to see how organizations described the platform teams’ roles and responsibilities. The results showed platform engineers commonly take the reins on everything from architecting solutions to tooling acquisition and internal evangelism. Here are the five top-cited roles for platform teams:
- Develop and standardize a cloud infrastructure strategy (95%)
- Architect cloud solutions (95%)
- Create and distribute cloud management, operational policies and best practices (94%)
- Determine which cloud vendors and/or technologies are used in production (93%)
- Define and measure site reliability (91%)
The Path to Cloud Maturity
In general, companies are still confident about their cloud trajectories. But, getting to a mature multi-cloud state will take work. One area that needs further scrutiny is cybersecurity, as password or credential leakage is now seen as the most prevalent cloud security threat. Organizations will require better DevSecOps best practices to avoid such leaks. That, and certain open source cloud-native tools could also aid security and compliance efforts.
A mature cloud strategy will also require localizing development efforts through shared platforms and increasing infrastructure automation. On that note, 83% of organizations say automation tools are important or extremely important to operationalize their multi-cloud infrastructure. More automation for cloud-native areas like CI/CD, observability and Kubernetes management could also abstract complexity and fill the skills gap.
As the HashiCorp study clearly showed, those organizations that have matured their cloud strategy are far better off because of it. They tend to understand their workflows better and can more seamlessly introduce new automation. Now, it’s only a matter of getting there …
-
2024—Year of the Linux Desktop? ChromeOS Reflects its Inner Penguin ¦ GNOME Rethink
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Can the Linux desktop installed base ever break the mythical 10% barrier? Google has been refactoring ChromeOS, and GNOME is working on new window manager ideas. (more…)
-
How Platform Engineering Bridges the IT and DevOps Divide
Ballooning lead and process time, with an opaque corollary relationship to fixed and marginal cost, is a significant and complex burden for any enterprise. And multi-cloud complexity and inefficient organizational and technical processes are some of the most wasteful areas companies face–so much so that Synergy Research reports organizations are cutting back on their cloud spend as tech budgets continue to fall under intense scrutiny, resulting in a Q1 2023 cloud spending growth rate of 19% compared to 32% just last year. With cloud cost containment a top priority, Forrester advised tech leaders to demonstrate to CFOs how they will stay on budget and demonstrate ROI at the board level. A key area where companies can eliminate expensive waste and increase time to value is the relationship and workflow between IT and developers, which, while being the DevOps charter for nearly 15 years, has gotten less attention among the viable cost-containment strategies for enterprises in 2023. Revisiting the latest iteration of this DevOps charter through the implementation of platform engineering strategies can help enterprises create a faster, leaner and more scalable dynamic. Let us examine why it’s now time for enterprises to consider platform engineering as a potent way to not only increase efficiency but also to bridge the divide between DevOps and IT.
The Old Way: Forcing DevOps to Dance to the ITSM Tune
In traditional enterprise organizations, IT typically dictates a top-down ITSM practice that limits the tools an organization can use for development and deployment; in many cases, it inhibits CI/CD and other core capabilities and practices of DevOps. This awkward attempt to accommodate modern DevOps practice through an aging ITSM paradigm has led to a suboptimal software delivery performance and poor developer experience (DX).
McKinsey, highlighting the criticality of DX, found developers at a leading SaaS provider spent 20% to 30% of their time integrating different services such as firewall changes, DNS and security access to deliver code through its CI/CD pipeline. That’s a lot of time spent on low-value, repetitive tasks, ensuring a painful journey for a disappointing result. If the DX pain is high enough, developers often go around IT and create resources on their own, leading to shadow IT, adding new cost burdens and growing security exposures. Every minute and engineer count: Five minutes saved per day among 500 developers saves a week’s development time.
Surely, there is a better way.
Platform Engineering: Collaboration Through Composition
Platforming engineering and “platform as a product” have been key to the PaaS ecosystem for years but are now gaining fresh traction in the industry. In Puppet’s State of DevOps Report, 51% of respondents said they had already adopted platform engineering and 93% said it was a step in the right direction. Gartner predicted 80% of software engineering organizations will have platform teams by 2026.
The concept can be defined in several ways. Gartner reported platform engineering is “an emerging trend intended to modernize enterprise software delivery… designed to support the needs of software developers and others by providing common, reusable tools and capabilities, and interfacing to complex infrastructure.” PlatformEngineering.org’s recent blog post defines it as the discipline of designing and building toolchains and workflows for self-service capabilities in software engineering organizations during the cloud-native era.
Regardless of definition, platform engineering is the latest iteration of IT centralization, though now attempting to retain all the benefits of distributed team empowerment through “composition” rather than converged control. As modernization continues, we experience the ebb and flow of composition and decomposition, both organizationally and technically. Microservices, Kubernetes, and GitOps patterns pushed the rational limits of decomposition and significantly raised the level of technical complexity in the enterprise. The natural counter pressure is a new cycle of composition that aims to simplify that complexity through platform engineering practices. Teams now look for these toolchains and workflows to be centralized in internal developer platforms (aka IDPs), exposing the development, deployment and maintenance of the entire life cycle of modern, cloud-native applications.
IDPs can facilitate the flow, speed and innovation DevOps teams are demanding without sacrificing stability, security and control. They enable developers to configure their toolchain while meeting all controls through approved blueprints or “golden paths,” reducing lead and process time. Developers can focus on delivering business value instead of infrastructure tasks.
Bringing IT and DevOps Back Into Alignment Saves Time, Money and Sanity
Platform engineering creates a collaborative framework that connects IT and DevOps teams, so they can focus on outcomes rather than methods. Developers can use an IDP to follow conventions that speed up development and prevent anti-pattern creep. Guardrailing choices through golden paths reduce cognitive load and simplify execution context for security teams. The result? Developers can focus on what they were hired for and want to do–creating innovative, transformative applications and differentiated value. Security can sleep well knowing bespoke behavior isn’t opening new attack surface areas.
On the Day 2 operations side, IDPs enable SRE and support teams to have a fleet-wide view and work effectively with developers to streamline the management of production needs across all applications, clouds and tech stacks.
The reframing of composition breaks the false dichotomies between IT and DevOps practices. You don’t have to choose between control and flow, change isn’t triggered by serial requests or async events, safety isn’t found in planning or speed, value isn’t found in stability or innovation and administrative plane primitives aren’t servers or clusters. Platform engineering practices allow you to prioritize composition, orchestration, golden paths, holistic scalability and fleet management to get teams out of entrenched myopia and embrace systems thinking that gets your business delivering strategic gain instead of tactical win/loss tallies.
Finally, platform engineering positively impacts software delivery and operational performance metrics by streamlining development processes and reducing manual efforts. The bottom line is faster time to market while ensuring more consistency, less human error and improved software quality. This maximizes productivity, minimizes downtime and optimizes resource utilization, leading to cost advantages and increased efficiency for enterprises.
-
Forking RHEL! Oracle and SUSE Join the Fight ¦ Silverman Sues AI Firms
Welcome to
The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: SUSE announces a Red Hat Enterprise Linux fork after Oracle said a similar thing, and Sarah Silverman says OpenAI and Meta have stolen her words. (more…)
-
Threads: Twitter Killer or Ad-Infested Hellscape?
Welcome to
The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Meta launches its much-leaked Threads microblogging app. A spinoff from Instagram, it’s already reached 30 million signups at the time of writing—even before it’s available in the EU. But how is Meta’s DevOps team making it scale? (more…)
-
IBM/Red Hat Sparks Anger at GPL ‘breach’ as RHEL Source Locked Up
Welcome to
The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Red Hat Enterprise Linux (RHEL) goes closed source. Well—no, not really; but IBM’s lawyers are accused of skating so close to the edge of legality that they risk violating the GNU Public License. (more…)
-
How Platform Engineering Protects Profit, People and our Planet
Platform engineering is emerging as a way of evolving DevOps, with 94% of respondents to the recent State of DevOps report agreeing that the practice helps their organization to reap the benefits of DevOps. The business arguments for platform engineering are clear: In that same report, respondents stated that platform engineering improves efficiency (59%), speeds up delivery time (58%) and improves workflow and process standards (57%).
But in addition to the huge benefits organizations are now realizing in terms of business value, platform engineering is also pretty crucial for a business’s long-term viability. And by that, I mean that platform engineering can be inherently sustainable.
How Can Platform Engineering Be Sustainable?
First, it’s important to note that ‘sustainability’ here doesn’t just mean ‘green.’ Rather, sustainability means that an organization is able to meet the needs of the present without compromising the ability of future generations to meet their needs, too. These are needs that come in three different varieties: People, planet and profits.
This means that to be truly sustainable, platform engineering needs to deliver lasting and stable returns on things like return on investment year-over-year, improved awareness of and reduction in an organization’s carbon footprint and better governance and optimization through monitoring and analytics.
Any approach which supports business growth while simultaneously empowering employees and being kind to the planet is the dream for pretty much every organization. But how exactly can departments and wider organizations unlock all of these benefits? The answer is: There’s no magic bullet.
Sustainable platform engineering is about treating your platform as an iterative cycle: It isn’t the actual creation of the tool that matters, as much as it is the maintenance, upkeep and usability of the platform as your business moves forward. This, naturally, will take time, effort and money—all of which, it’s safe to say in the current climate, are in short supply. But it is crucial that businesses act strategically, looking for areas for improvement and paths that will all add up to a more sustainable business in the long term.
The Three Paths to Success
These paths can be divided into three main areas of focus, and the first is GitOps or infrastructure-as-code (IaC).
A tweak that every organization should start making is using Git as a single source of truth for deployments, enabling developers to work faster by empowering them to use their own code without hands-on involvement from ops teams. This could be combined with optimized infrastructure, moving toward IaC to enable faster and more efficient deployments.
These principles form the foundation of sustainable platform engineering as they protect profits, allowing the business to keep moving forward and investing in the other areas crucial for a sustainable operation. Combining IaC with GitOps ensures speed and simplicity while maintaining a high-quality end product.
Creating an optimized infrastructure is important, but just as important is empowering employees to feel good about the work that they are doing. That’s why implementing a self-service tool alongside an optimized UX is the second area of focus. This removes the pressure on those in the tech department to be infrastructure experts while erasing the need for endless tickets to be raised for issues.
By freeing up DevOps engineers and platform experts to spend more time doing what organizations need them to do, businesses can keep innovating to maintain an edge over their competitors. Self-service capabilities combined with an accessible UX are a useful combination for a much-improved developer experience (DevX).
Finally, it’s important to keep in mind that self-service isn’t just going to benefit end-users; it also has major upsides for improved governance, as well as keeping control of cloud spending and carbon footprint impact. Optimizing and iterating your platform engineering tool means that a business can keep up with the fast pace of the global tech market while also continuing to implement the latest tweaks and upgrades. It all adds up and builds on the organization’s platform engineering solution.
Reaping the Rewards of Sustainability
If an organization follows the above three paths, it is well on its way to a more sustainable business powered by platform engineering. By focusing on targeted, achievable projects to build a solid foundation, organizations can achieve big benefits for people, the planet and profit.
From an economic perspective, sustainable platform engineering ensures legacy infrastructure is modernized, silos are broken down and production is accelerated, all ensuring the business doesn’t risk lagging behind competitors. A stable and secure return on investment is key to sustainability in the medium to long term, and sustainable platform engineering ensures a business is able to weather future challenges and keep a healthy profit margin.
As we all know, underpinning any great business is the people who work there. Techs get stressed by having to interact with infrastructure they know nothing about, DevOps and specialists get stressed when those techs desperately raise tickets in an attempt to get help, and platform engineers get stressed when the weight of providing a solution to these problems rests squarely on their shoulders. Implementing a UX-strong self-service portal relieves the pressure, allowing employees to focus on delivering real value to the business through innovation. A better DevX ultimately leads to better business outcomes.
And, of course, it’s ultimately no good having a successful business if the planet we all share continues to suffer from the effects of the climate emergency. Just like a business that hemorrhages money is unsustainable, so is a planet that wastes its finite resources. By getting control of cloud usage and controlling cloud climate impact, businesses will not only be empowering their people and improving their bottom line, but they will also be making a tangible difference to our shared environment, supporting our planet as it recovers from years of climate neglect.
Platform engineering offers lots of tangible benefits for businesses, but sustainable platform engineering can be even better, not just for a business, but employees and our shared environment. It might just be time to upgrade DevOps and start thinking about building a sustainable organization that supports profit, people and the planet.
-
How Can DevOps Evolve? Platform Engineering
Kaylee Van De Voorde co-authored this article.
Has your team adopted a DevOps mindset? Are your engineers using tools like Docker containers, Kubernetes and Terraform? Over the last decade, many engineering teams that adopted DevOps culture and pivoted to cloud-native applications that required rapid scaling may face a new set of problems.
DevOps
DevOps practices have brought operations and software engineering teams together throughout the entire development life cycle, from initial concept through the engineering process to production support. This collaboration paved the way for iterative development: Improving velocity, providing rapid feedback and consequently improving application quality by uncovering bugs and environment requirements earlier without user impact. DevOps automation created a more “blameless” culture that improved trust, collaboration and innovation, which boosted team morale and increased engagement.
That said, DevOps teams have encountered many obstacles, including fears of job obsolescence and steep learning curves for every member. Teams that learned how to implement DevOps practices on their projects may have moved more slowly at first while learning how to work together in new ways and adopting new tools.
What is Platform Engineering?
So, what is platform engineering? Platform engineering bridges the existing gaps between software, hardware and infrastructure that come with adopting DevOps practices: Automated horizontal scaling, service discovery and cost optimization. Platform engineering makes it even easier and simpler for engineers produce high quality applications quickly by standardizing CI/CD deployments, ensuring that infrastructure is auditable and monitored, automating deployment methods and processes, providing accurate documentation and incorporating FinOps.
Platform engineering teams can make these improvements available to their teams by creating tools such as an internal developer platform (IDP) that includes APIs that expose internal tools and features within the platform.
The Next Evolution
Evolution does not mean starting from nothing.
The core tenets of DevOps practices still hold their value; aligning software engineering and operations teams to better create applications that embody business and user functionality and run smoothly within an incorporated infrastructure, automation, security and dedicated site reliability engineering (SRE). Many DevOps teams may already be moving toward an overarching platform engineering culture and simply need to refine their standards. Distinct application teams may already be building similar features to provide platform functionality; a strong indication that a platform engineering team would be beneficial.
Choosing to create a dedicated platform engineering team has tradeoffs: Building a platform engineering team pulls resources from building applications and new features that the business needs, but application teams could experience an increase in productivity if a separate platform engineering team is created. A dedicated platform engineering team can also provide clear, unified direction and nuanced goals for application teams; this is incredibly important when creating standards and internal tools.
A platform engineering team is just one way that to advance and improve engineering culture. DevOps teams that may not require platform engineering can continue their growth by refining their existing practices and patterns in their everyday work.
-
Survey Sees Value But Limited Understanding of Platform Engineering
Platform engineering is emerging as method for managing DevOps at scale, but a survey suggested that many organizations are still not clear on the concept.
A survey of 100 DevOps engineers, platform engineers, developers, site reliability engineers and IT managers conducted by Civo, a cloud service provider, found 85% of respondents work for organizations that have partially or fully embraced a DevOps culture, with nearly two-thirds (65%) identifying platform engineering as the next evolution of DevOps. Nearly, a third (30%) of respondents work for organizations that are already currently evaluating whether they should adopt platform engineering.
Overall, the survey found 50% of the respondents at organizations that have adopted platform engineering reported improved standardization, with 43% also seeing faster product delivery.
However, only 41% of the respondents in DevOps roles that were familiar with platform engineering said they had a moderate to high level of familiarity with the concept.
Saiyam Pathak, director of technical evangelism at Civo, said the survey makes it clear that, in terms of understanding how platform engineering best practices should be implemented, there is need for more clarity. Many organizations initially adopted DevOps to spur innovation, so any approach to centralization shouldn’t come at the expense of developer productivity, he noted.
Historically, DevOps adoption of DevOps workflows in most organizations has been driven from the bottom up. As a result, it’s fairly common for multiple application development teams within an organization to have adopted multiple DevOps platforms to manage workflows. Advocates for platform engineering are making a case for eliminating many of those redundant platforms in favor of, for example, a single continuous integration/continuous delivery (CI/CD) platform that serves as a corporate standard.
At the core of those efforts is the assumption that centralization will reduce costs by eliminating redundancy. At a time when more organizations than ever are sensitive to the cost of IT, there is always going to be a natural tendency to rationalize platforms. Challenges arise, however, when centralization results in a loss of flexibility. Many of the teams adopt DevOps because they want to be able to add and replace tools as they see fit. If there is a corporate standard that narrows their tool options, those teams may start to look for ways to do an end-run around the corporate standard.
The philosophical IT battle between advocates for centralization and proponents of decentralization has been around since the first PCs showed up in enterprise IT environments. As always, organizations need to strike a balance between both extremes, said Pathak.
It’s not clear how much platform engineering might accelerate the pace at which applications are built and deployed, but the total cost of DevOps should decline as various tools and platforms are consolidated. The challenge, of course, is that centralizing DevOps workflows is likely to disrupt the application development workflows on platforms that may about to be rationalized.
Regardless of what platforms become the corporate standard, the one thing that is certain is the cultural challenges associated with successfully making that transition will loom large within most organizations.