devopsy.kinsta.cloud

Tag: platform engineering

  • Platform Engineering: Creating a Paved Path to Reduce Developer Toil

    Platform Engineering: Creating a Paved Path to Reduce Developer Toil

    This article is part one of a series of three on the boom in platform engineering and its role in reducing cognitive load and creating a better developer experience.

    Developer toil and cognitive load in the cloud-native space is real. The complexity posed by microservices, Kubernetes, and “software-defined everything” almost necessitated that ops needed to solve many of these issues to support cloud-native development.

    Developers taking on full dev life cycle ownership and the requirement to shift left have resulted in cross-functional and operational concerns that have added even more complexity to already challenging workloads and unclear workflows.

    To fuel productivity and consistency and enable developers to build and ship apps faster and more securely, one must understand what a ‘paved path’ might look like and where responsibility for different aspects of the development life cycle resides.

    Supporting the Majority of Developers Without Alienating the Rest

    Most developers want and need paved paths, particularly as paradigms shift; the adoption of cloud-native development being one such shift. These paths must help them reach the “safety” and productivity on which they are measured in their daily output and provide focus for completing their tasks. Freedom to make choices about processes and tools is not a priority because it ends up being noise that adds to the cognitive load and, in most cases, does not serve the company’s software shipping goals. This majority needs tighter guardrails and guidelines to do their work and get their code into the world without having to figure out too many operational parameters for themselves.

    The cloud-native tooling landscape is vast, duplicative and overwhelming. Just trying to select and test tools could be a developer’s full-time job. Likewise, many organizations and developers have reported frustration—they are wasting a lot of time on repetitive work and routine tasks that deliver no business value, such as setting up environments and troubleshooting CI pipelines along with slow feedback loops in the development process.

    As organizations increase pressure to deliver more with less and do so more frequently, these responsibilities make for a poor and unproductive developer experience and a far-from-ideal use of organizational resources. Stack Overflow’s 2022 Developer Survey reported that 62% of respondents spend more than 30 minutes a day fixing problems and looking for answers, leading to somewhere between 333-651 hours of lost time per week across a team of 50 developers. This reinforces a need for upfront investment in the paved-path developer platform, which eliminates much of this frustration.

    It’s worth noting there is also a so-called 1% developer–one who wants to experiment and discover new tools on their own. Defining the developer experience within narrow guardrails ends up stifling this minority, which not only hampers their productivity but could stand in the way of innovation for organizations as a whole.

    Building quality software requires accommodating both kinds of developers. For most development teams, the goal is to give them a clear route to building apps faster and shipping safely by prescriptively paving the path. The path also must provide enough freedom for adventure, allowing developers to stray from the path (within reason and where it makes sense).

    Internal Developer Platforms: Paving Just Enough Path With Platform Engineering

    Platform engineering and internal developer platforms (IDPs) are much-touted approaches to taming the toil. It is a part of the solution, and as it gains traction, it is becoming more mature. That is, instead of developers and development teams taking on full responsibility for every phase of the software life cycle and potentially cobbling together something resembling a developer platform, the industry trend is moving toward more comprehensive, commercial out-of-the-box platforms. Whether or not this addresses the actual needs of the development organization, this shift works because developer experience is now a business-critical requirement, and a centralized developer platform facilitates that experience (with flexibility enough to house all types of developer experience and developers working across industries).

    The developer platform concept, supported by architecture, DevOps and SRE teams, contains the ingredients of transformation both for business and for developer experience. As the concept reaches critical mass and the “productifying” of developer platforms matures, one thing seems likely: By shifting to a platform model, development organizations are moving from ops-first to developer-first Kubernetes. The developer experience becomes defined by its optimized tools and workflows (and enables wider adoption of cloud-native development across larger organizations and broader industries).

    Refocusing Developer Efforts: Abstractions, not Distractions

    Cognitive toil will exist for developers regardless of the tools they’re offered. Technologies grow more complex and change frequently, and there are performance metrics that must be met. Taking the focus away from developers’ primary concerns—coding and shipping software and features—contributes to this toil without generating much return. As much as developers should be able to gain enough understanding of the way their code behaves and the consequences it can have downstream when shipping and running it, they should not need to become DevOps experts as a part of their work. This is why DevOps, despite predictions to the contrary, isn’t dead.

    The paved-path developer platform gives developers exactly that: The ability to shift left without drifting right. The platform provides much-needed abstractions and tools that enable developers to refocus on priority work without distracting or overwhelming them with too many choices.

    Becoming dev-first doesn’t kill off ops, either. Dev-first Kubernetes and a centralized developer platform elevate the importance of the developer experience while continuing to highlight the importance of DevOps in supporting and optimizing the developer experience. This is an entire topic in and of itself, which will be explored in part two of this article series.

  • Where Does Observability Stand Today, and Where is it Going Next?

    Where Does Observability Stand Today, and Where is it Going Next?

    As we move into the second half of 2023, it’s only natural to engage in the annual ‘state of the industry’ reviews and take a general inventory of the past year while looking out into the immediate future. Observability has gained traction among DevOps and platform engineering teams over the past year and is poised to continue that velocity. But my observation is that the current observability model is broken.

    Too Much Data

    Long story short: Many organizations (and engineers specifically) agree that they are sending too much data to their monitoring tools, resulting in an inability to garner valuable insights because the systems are massively overloaded. Sending this high volume of data also costs too much and requires an inordinate amount of time for analysis, which many organizations don’t have. Put simply–more data doesn’t mean more clarity; in fact, we see the opposite.

    Massive adoption of cloud-native technologies such as Kubernetes that heighten the challenge of efficiently monitoring environments is also accelerating this problem. Beyond increasing costs and complexity, many teams seem to feel that their current approach doesn’t create the desired level of observability for Kubernetes, full stop.

    Another major issue we see on the observability landscape relates to the overall maturity of monitoring processes and teams. In many instances, even when organizations have effective tools, they struggle to interpret the data produced into targeted conclusions that their teams can translate into effective actions. And this is further amplified by the outstanding engineering talent shortage because good people are harder to find and retain than ever before.

    We also see a lot of pervasive challenges related to organizations’ desire to use popular open source monitoring tools-especially if they are attempting to do so in an integrated manner or to support rapidly growing deployments at scale.

    Overwhelmed by Observability Data?

    In summary, if you’re overwhelmed by observability data, if complexity and costs grow out of control and you’re also struggling to make sense of the available information and improve your cloud applications, we think that means that the current observability model really is broken.

    To that end, gaining a deeper perspective into end user opinions on these topics, among others, is the precise goal of the annual DevOps Pulse Report, currently in its sixth year and now in the survey stage. We predict that this year’s research project will underline the aforementioned challenges and help frame how teams might seek to address them.

    What types of findings do we hope to unearth? Last year, respondents revealed that despite a growing perception of DevOps maturity, it was actually taking them more time to respond to emerging problems. Roughly 64% of respondents reported that MTTR during production incidents was over an hour, compared to only 47% the previous year.

    Last year’s report also highlighted the reality that monitoring complex environments remains difficult with over 52% of respondents citing Kubernetes, microservices and serverless as their primary challenges in gaining improved observability.

    Both of these previous findings also seemed to underline the growing issues of data overload and spiraling complexity. We’ve also seen previous evidence of a lack of maturity around distributed tracing—a critical observability practice aimed directly at increasing visibility into complex architectures—despite the fact that 75% of respondents listed it as a priority in last year’s research.

    Is your team struggling to make sense of its available monitoring data, or do so in a way that aligns with your current processes or budgets? Hopefully, the 2023 DevOps Pulse Report, set to publish early this year, will put some numbers behind these assertions and help us all understand how to move forward.

    If you want to participate, you can do so here.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers. Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    Apple
    San Diego, California
    Site Reliability Engineer (SRE) – Video
    $115,000 to $217,500

    SimplyHired

    ScientiaMobile
    Reston, Virginia
    Senior DevOps Engineer
    $110,000 to $150,000

    Monster.com

    Larsen & Toubro Infotech Ltd.
    Farmington, Connecticut
    DevOps Engineer
    $110,000 to $130,000

    Indeed.com

    TherapyNotes.com
    Remote
    Senior DevOps Engineer
    $105,000 to $155,000

    Dice.com

    MRoads
    Dallas, Texas
    Platform Engineer – SRE
    $100,000 to $120,000

  • Automation Challenges Holding DevOps Back

    Automation Challenges Holding DevOps Back

    A survey of 500 IT and engineering leaders in the U.S. published today finds organizations build an average of 6.7 internal applications per year themselves in addition to investing in 7.7 separate platforms to build applications.

    Commissioned by Internal, a provider of platforms for building application development environments, the survey also found organizations are spending 40% of their time building and maintaining these internal tools and workflows. While more than three-quarters (76%) of respondents said that this work is necessary, it distracts from building applications that drive business revenue. A total of 82% said a lack of true flexibility and customization is the top reason so much time is spent on building and managing internal tools and workflows.

    A full 94% of respondents said they want a more collaborative platform that enables technical and non-technical users to build internal tools/workflows faster together. The survey also found 91% of respondents believe a platform that encompasses application development and workflow automation would save development time.

    Internal CEO Arisa Amano said one thing that holds IT teams back is that managing the platforms used to build applications requires too much programming expertise. Internal is making a case for a graphical approach to building application development environments so IT administrators can easily construct them using a drag-and-drop interface on behalf of an application development team. That approach makes it possible to allocate more programming resources to building applications, she said.

    More organizations than ever are revisiting how applications are built as the number of projects increases despite an uncertain global economy, noted Amano. Organizations of all sizes are reluctant to hire additional full-time engineers even as digital transformation initiatives demand a growing number of custom applications, she added. The only way to effectively address those conflicting goals is to adopt a platform that automates much of the workflow associated with building an application development environment, said Amano.

    It’s not clear how willing DevOps teams are to replace the platforms they use today to build those environments, but many organizations are still relying on bespoke platforms they built and continue to maintain. Many of those platforms are made up of a collection of tools that DevOps teams cobbled together over the years. However, many of the software engineers that initially built those integrations may have since moved on and the documentation left behind is likely to be scant, noted Amano.

    Most DevOps teams in 2023 will continue investing in higher automation levels to accelerate application development. The quandary they all face is whether to continue to invest in the platforms they have already or migrate to a platform that makes it simpler to build and maintain an application environment using automated workflows built and maintained by a vendor. Regardless of what they choose, it’s certain the days when organizations could afford to throw engineers at every integration challenge have come to an end.

  • GitLab Strengthens Remote DevOps Management

    GitLab Strengthens Remote DevOps Management

    GitLab this week moved to provide remote developers with access to a more secure continuous integration/continuous delivery (CI/CD) platform while at the same time adding a dashboard that makes it simpler to track the overall status of application development projects.

    David DeSanto, chief product officer for GitLab, said GitLab Remote Development takes advantage of an instance of the GitLab CI/CD platform running on a Kubernetes cluster. That cluster is accessed via a Web-based integrated development environment (IDE) to eliminate the need for developers to create development environments on their local desktops.

    That capability reduces developers’ time and effort on tasks other than code and gives organizations more centralized control over the intellectual property being created, noted DeSanto. Organizations will no longer have to worry, for example, about theft of a developer’s laptop that has sensitive data on it, he added. In addition, organizations will be able to more easily create “golden” images that can be consistently accessed by developers residing in multiple locations.

    In addition to making it easier to centrally manage a distributed team of developers, GitLab is also adding a GitLab Value Streams Dashboard that provides senior executives with access to metrics that assess the overall health of an application development project. While GitLab has always provided metrics to DevOps teams, DeSanto said this dashboard, available as a private beta, makes it possible to identify, for example, bottlenecks that might result in delayed delivery of an application.

    Value stream management (VSM) is, of course, not a new idea. The issue that DevOps teams are trying to come to terms with is whether they need to adopt a dedicated VSM platform. The other option is to simply provide a set of dashboards that enable senior IT and business executives to track projects’ progress without deciphering a set of much lower key performance indicators (KPIs). Previously, the GitLab platform surfaced a lot of nitty-gritty details that don’t provide the context senior executives required, noted DeSanto. The Value Streams Dashboard metric surfaces KPIs based on DevOps Readiness Assessments (DORA) metrics from Google alongside other flow metrics.

    These latest offerings are part of a larger case being made for an integrated DevOps platform that reduces the total cost of application development by eliminating the need for organizations to integrate various tools themselves. Each DevOps team can employ as many of the modules GitLab makes available as they need. Once invoked, they take advantage of the same data store as every other element of the GitLab platform. That capability is becoming more critical as organizations embrace platform engineering to reduce the cost of application development using DevOps best practices, said DeSanto.

    It’s not clear how much interest there is in replacing legacy CI/CD platforms that, in some instances, have been in place for nearly a decade. However, as the pace at which applications are being built and deployed continues to accelerate, there is no doubt that organizations of all sizes will need to modernize application development processes.

  • State of DevOps Report Finds a Rise in Platform Engineering

    State of DevOps Report Finds a Rise in Platform Engineering

    The speed of software development velocity is increasing, and to enable this, platform engineering is emerging to evolve the DevOps practice. Interest in platform engineering is rising—so much so that Puppet emphasized platform engineering in its 2023 State of DevOps Report.

    A whopping 94% of respondents agreed that platform engineering is helping their organizations better realize the benefits of DevOps. The move toward platform engineering has increased over time, bringing benefits such as increased release velocity, reliability and efficiency.

    Below, we’ll highlight some of the key takeaways from the report. We’ll define what platform engineering is and highlight its benefits. We’ll also consider obstacles on the path to embracing a platform-based development culture.

    Platform Engineering Supplants DevOps

    DevOps has been in use for a decade or more at most institutions, maturing steadily with new automation and infrastructure abstraction. The latest evolution of DevOps is a practice known as platform engineering—51% of organizations have already adopted this practice within the last three years and 93% said it’s a step in the right direction.

    Platform engineering is the discipline of designing and building self-service capabilities to minimize cognitive load for developers and to enable fast-flow software delivery.

    In essence, platform engineering is the process of creating standardized, self-service infrastructure that helps developers get up and running quickly. These self-service platforms typically handle capabilities like deployment (43%), operation of infrastructure (43%), monitoring production applications (42%) and security and compliance (42%).

    These shared internal platforms are maintained by a central platform team that embraces a product mindset to iterate the technology based on user feedback.

    State of Platform Engineering

    Platform engineering is relatively new— 27% of platform teams have formed within the last two to three years. Notably, big tech companies hopped on the bandwagon years ago, and non-tech companies are now following. Regardless, 70% believed their timing was “just right.”

    The goals of a platform engineering team are numerous. According to the Puppet report, 54% said problem-solving is the top goal. This is followed by educating and empowering developer teams (47%), setting best practices (46%) and optimizing iteration (44%). A platform engineering team’s scope of work is primarily focused on automating workflows and building and managing infrastructure.

    The ‘correct’ number of internal platforms is a moving target. The majority of organizations (57%) have between two and four platforms, and 30% have five or more internal self-service platforms. This demonstrates that although platform engineering is somewhat centralized, it’s also flexible to the needs of an organization.

    Benefits and Hurdles

    So, what are some benefits as opposed to traditional DevOps? Well, the most common benefit is improved system reliability, at 60%. This tracks, as a standardized platform could reduce human error and missteps in the development workflow. Following this benefit is improving efficiency (59%), speeding up delivery time (58%) and improving workflow and process standards (57%).

    The most common driver of adoption is increasing the speed of delivery. Slight more than half of respondents (51%) said development speed has somewhat increased since the inception of the platform team, and 42% say it’s increased by “a great deal.” Other top drivers include the need to scale, free up engineering time, control infrastructure spending and avoid duplication across teams. Those with more mature adoption tend to be more satisfied with the result, indicating that results increased over time.

    Of course, no new technological introduction comes without its hurdles, and platform engineering is not immune from certain pain points. For example, 34% said that cycle time is slower than expected. This might be due to nascent industry adoption and a lack of best practices.

    Just under one-third (32%) also noticed resistance to platform team adoption, and 32% reported a lack of communication around changes. Organizations will likely have to increase transparency and reduce siloing to encourage productive platform adoption that satisfies the needs of various stakeholders.

    According to the report, product management skills can help realize these goals. Some of the most fundamental product management skills here include strong communication skills (61%), problem-solving expertise (60%) and the ability to foster collaboration (54%). Many companies will also have to convince senior management to invest in platform engineering—nearly half agreed that senior management doesn’t yet understand its value.

    Future Outlook

    Confidence around platform engineering is increasing, especially for teams with mature DevOps approaches. As a result, the practice will likely gain more traction in the coming year—71% of respondents reported their firm plans to hire people with platform engineering experience over the next 12 months.

    Over the next year, organizations that adopt the practice will want to increase awareness of platform capabilities and set expectations across the organization. This will also include setting governance policies and satisfying the needs of specific developers, departments and the organization as a whole.

    Above, we’ve scratched the surface of the current state of platform engineering, its benefits and its pain points. For plenty of other insights, you can pick up a copy of The State of DevOps Report 2023: Platform Engineering Edition from Puppet by Perforce here.

  • Career Trajectory

    Career Trajectory

  • Best of 2022: SRE Vs. Platform Engineering: What’s the Difference?

    Best of 2022: SRE Vs. Platform Engineering: What’s the Difference?

    As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    Site reliability engineering (SRE) teams and platform engineering teams share similar goals—like maximizing automation and reducing toil—and similar methodologies. But they have different priorities, and use somewhat different tools to achieve them.

    What are SREs? What are platform engineers? How is each role similar and different? This article will answer all your questions.

    Responsibilities of SREs and Platform Engineers

    The main responsibility of site reliability engineers is ensuring that IT systems are reliable, which means that the systems meet performance requirements. Part of their job includes working with developers and IT operations teams to maximize the reliability of applications as they move down the software delivery pipeline.

    Meanwhile, platform engineers focus on managing and optimizing the software delivery process. You could think of this role as one that translates DevOps principles into practice by finding ways to implement continuous delivery, continuous improvement and other DevOps priorities.

    Thus, both SREs and platform engineers play a role in managing the software delivery process and finding ways to make it as smooth and efficient as possible. They also share a common enemy—toil—in the sense that both groups seek to avoid manual, inefficient processes by automating as much as possible. SREs automate complex incident response operations using tools, for example, while platform engineers might deploy methods like GitOps to automate some aspects of software delivery.

    Differences Between SRE and Platform Engineering

    That said, SRE and platform engineering are distinct roles. The key differences boil down to:

    • Goals: The main job of SREs is maximizing reliability, which is different from optimizing the speed and efficiency of software delivery operations. Theoretically, you could have a highly reliable system even if your software delivery chain is manual and slow.
    • Scope of role: Software delivery operations are the main focus of platform engineers. SREs care about software delivery, too, because the way applications are created plays a role in reliability and fixing issues. But SREs also have many other responsibilities, like incident response and helping to manage infrastructure, which fall beyond the scope of software delivery.
    • Tools: SREs rely on tools that help them discover and solve various types of reliability issues. In contrast, platform engineers leverage tools like source code managers and continuous integration (CI) servers to help automate software delivery.

    So, although SREs and platform engineers may use similar methodologies and possess similar skills, their day-to-day operations tend to look quite different.

    Should You Pursue an SRE or Platform Engineer Career?

    Given the skills overlap between SRE and platform engineering, some engineers are a good fit for either role. But which one is better, from a career perspective?

    Salary-wise, there’s not a whole lot of difference. Glassdoor pegs the average salary of platform engineers at about $115,000 compared to $127,000 for SREs. SREs may earn a little more (and it’s worth noting that SRE salaries can vary widely), but the difference in pay is not as great as that between, say, IT operations engineers (who make a mere $77,000, according to Glassdoor) and SREs.

    The roles are also similar in terms of demand. Unlike software engineering or even DevOps engineering, SRE and platform engineering are both roles that have begun to become popular only in recent years. SRE as a concept has been around for a while—it originated at Google in the early 2000s—but it wasn’t until recently that most companies began seeking SREs for their teams. Likewise, platform engineering didn’t become a thing until the advent of microservices and cloud-native everything pushed companies to devote more engineering resources to manage their increasingly complex software delivery processes.

    What this means is that, at least for now, it’s easy to find jobs in SRE and platform engineering as long as you have the right skills. Many companies are eager to hire for both types of roles.

    Choosing the role that is right for you, then, depends mostly on your preferences and how you like to work. The life of a platform engineer is likely to be somewhat more predictable than that of an SRE because platform engineers don’t have to take the lead in managing incidents when something breaks at 2:00 a.m. On the other hand, SREs enjoy a more diverse set of responsibilities, which can make the job more exciting—especially if software delivery tools like Git or Jenkins don’t make you excited to get out of bed in the morning.

    Conclusion

    Platform engineers and SREs play central roles in automating complex processes and eliminating toil. But they tackle these issues from different angles and they use different tools. The right role for you depends on what you enjoy doing most.