devopsy.kinsta.cloud

Category: Platform Engineering

  • How IT Services Can Embrace Platform Engineering

    How IT Services Can Embrace Platform Engineering

    We are all familiar with DevOps as a newer approach to software development. The concept is designed to enable developers and IT operations teams to work together. Platform engineering, on the other hand, is a more mature, older approach that has become new again.

    Platform engineering, at its core, is the concept of productizing your internal development platforms to make it easier for developers to work. It is an emerging technology approach that can help accelerate the delivery of applications and the pace at which they produce business value. More recently, companies have grown their complex software architectures which have created silos for the teams that manage these various architectures. There may be multiple teams with various specialized skill sets and a single DevOps engineer who takes care of the continuous integration (CI) jobs to deploy and release to production.

    Now let’s imagine that you have 10 squads that focus on building and maintaining different applications. Now all of these teams are building and operating 10 different platforms to run their applications, which creates inefficiency across the teams and reinforces silos. In this scenario, teams may spend half of their time building and maintaining the platforms instead of delivering business value. This creates a large cognitive load on developers.

    This model becomes very hard to scale. Even though you have these squads managing their own infrastructure and platforms, you may find gaps because there is no centralized standard. So how do platform engineering concepts help with these complex landscapes and boost efficiency?

    The Consulting Mindset

    Platform engineers take all of the tools used to run an application across your various teams and standardize their usage. This enables self-service capabilities for your engineering organizations. Platform engineering can help centralize how you are deploying and running your applications across a complex software landscape. This includes not only the standardization of tools being used but also documentation and processes.

    Establishing platform teams requires a consulting mindset and thinking about what resources, services and components can be centralized to better serve the rest of the organization. Some may refer to this as the center of excellence (COE) model, or simply IT for IT. This boosts efficiency and cuts down the time to delivery by providing a central set of tools and platforms for your engineering organization. Many may see platform engineering as a cost center, but the ROI is higher-quality releases, a standardized way of developing and delivering software and shorter delivery times.

    Applying This Concept to IT Services Firms

    In my consulting days, I worked for firms that focused on delivery for the Salesforce platform. We often had complex projects that we needed to deliver for large enterprise customers who had multiple applications integrating into their CRM. Each project team often had a different way of delivering and developing based on the customer’s needs. This created specialized skill sets, but we had no standard way of developing and delivering. Investing in a platform engineering team to support the delivery organization can add immense benefits for large consulting firms, enabling them to standardize the tools and platforms being used to run and deliver applications at scale.

    Having a centralized place for your developers to go for process, documentation and tools to help them build and run applications can increase the speed at which they deliver. The efficiencies gained in having a smoother self-service flow for your developers and cutting down the delivery time are worth the investment. It brings significant long-term value that the customer will experience long after the engagement is completed, which can greatly impact future considerations for your firm.

    IT services firms should find ways to align and centralize their technologies with platform engineering concepts. You may not be able to exactly build out an internal developer platform, but adopting some of the core concepts will help you be more efficient as an organization. For example, if you are an IT services firm specializing in delivering applications on the Salesforce platform, look for extensible DevOps platforms that can support your development organizations.

    Also, look for ways to enable a self-service workflow to make it easier for your developers to work and scale. This may include investing in a platform engineering function to support the rest of the delivery organization. By establishing platform engineering concepts, you will be able to cut down the delivery time and create consistency in how you deliver and deploy applications for your customers.

  • Cycloid Adds Templates to Simplify GitOps Workflows

    Cycloid Adds Templates to Simplify GitOps Workflows

    Cycloid today added a Bootstrap Stacks capability that provides DevOps teams with the ability to build Git templates or use ones created by the company to define cloud computing environments.

    The capability is accessed via a portal added to the Cycloid platform for automating software delivery. It also makes it simpler for DevOps teams to reuse templates that are compatible with Terraform or Kubernetes environments as needed.

    In the months ahead, Cycloid plans to release Headers, a metadata tool that automatically populates project parameters in display pages and a project management tool to provide a unified view for each development environment, including continuous integration/continuous delivery (CI/CD) pipelines, documentation, automation, logs, cloud costs and carbon emissions.

    Cycloid founder Benjamin Brial said the templates provide a layer of abstraction that makes it simpler to implement GitOps workflows. As a best practice, GitOps enables DevOps teams to unify the management of code to provision IT infrastructure and software delivery. However, providing access to a Git repository alone is insufficient, said Brial. DevOps teams need access to a set of templates that automate the process and eliminate what is otherwise a complex set of tasks, he noted.

    Easily discovered templates also eliminate organizations’ tendency to create redundant automations simply because a development team was unable to find an existing framework that another team already created, added Brial.

    That approach also enables DevOps teams to more easily embrace platform engineering principles to centralize the management of DevOps using a Bootstrap Service catalog embedded within the Cycloid platform, said Brial. That capability makes it possible to enforce GitOps policies in a way the developers will have to respect because the automation frameworks are readily available, he added.

    It’s not clear whether organizations are embracing platform engineering. Historically, adoption of DevOps workflows in most organizations has tended to flow from the bottom up. As a result, it’s fairly common for multiple application development teams within an organization to have adopted multiple DevOps platforms to manage workflows. Advocates for platform engineering are making a case for eliminating many of those redundant platforms in favor of a corporate standard.

    At the core of any approach is the assumption that centralization will reduce costs by eliminating redundancy. Issues can arise, however, when centralization results in loss of flexibility. Many of the teams that adopt DevOps want to be able to add and replace tools as they see fit. If there is a corporate standard that narrows their tool options, it’s just a matter of time before those teams start to look for ways around the corporate standard.

    Each organization will need to strike a balance between flexibility and the need to bring some order to DevOps workflows. In the meantime, however, the pace at which multiple software development initiatives are being launched is likely to force a platform engineering conversation that, in many cases, many organizations have postponed longer than anyone cares to admit.

  • Quali Adds Governance and Cloud Cost Control Tools to Automation Platform

    Quali Adds Governance and Cloud Cost Control Tools to Automation Platform

    Quali today revealed it added governance and cloud cost management tools to its Torque platform for automating IT infrastructure management.

    David Williams, senior vice president for market strategy at Quali, said the added governance tools will make it possible to enforce security, compliance and consumption policies as IT environments are launched using the automation blueprints that Torque enables IT teams to create.

    At the same time, IT teams can now also define consumption policies based on cloud runtime or regions employed within a cloud computing environment. In addition to taking advantage of dashboards to visualize consumption of cloud resources, IT teams can also use Torque to identify cloud waste including detached, inactive and overprovisioned instances. They can pause, right-size or tear down environments to reduce costs and eliminate the possibility of a billing surprise at the end of the month, noted Williams.

    Torque also enables IT teams to detect drift and then update live cloud environments to remediate any that fall out of scope, he added.

    Quali has been making a case for an environment-as-a-service approach to managing IT. That approach is based on a set of triggers for controlling infrastructure via a software-as-a-service (SaaS) platform that can be invoked either via an application programming interface (API) or a graphical user interface (GUI).

    In addition to providing a set of infrastructure provisioning guardrails that prevent mistakes from being made, Torque makes it possible to automatically deploy complete application environments on top of infrastructure provisioned as code using blueprints rather than relying on low-level Terraform scripts. The goal is to increase developer productivity by enabling IT teams to create self-service portals that developers can use to spin up environments.

    That approach is becoming more appealing as many organizations begin to adopt platform engineering as a method to centralize the management of DevOps workflows, noted Williams. In fact, as part of that effort, many organizations are starting to unify the management of cloud and on-premises IT environments, he added.

    While many developers routinely provision IT infrastructure using tools such as Terraform, many of them lack the cybersecurity expertise required to ensure cloud computing environments are configured properly. Developers also tend not to remember to de-provision cloud resources when no longer needed; over time, those factors contribute to unnecessarily increased cloud costs.

    Ultimately, the goal is to find ways to centrally manage IT environments to reduce costs and improve cybersecurity without inhibiting the pace at which developers are building innovative applications, said Williams.

    At the same time, as application environments become more complex, it is becoming even more challenging to provision and manage IT environments at a time when many organizations are reluctant to hire additional IT staff, he noted.

    One way or another, IT environments will become more automated. The issue is determining how best to organize the IT teams managing those environments as tasks that once required a lot of manual effort are increasingly automated.

  • Pulumi Embeds Terraform Conversion Tool in Platform

    Pulumi Embeds Terraform Conversion Tool in Platform

    At its virtual PulumiUP event, Pulumi this week announced it embedded an ability to convert instances of Terraform into its infrastructure-as-code (IaC) platform via a simple command line interface (CLI) command. In addition, Pulumi added a Review Stacks capability that automatically creates a dedicated cloud environment to review code every time a pull request is opened.

    Pulumi is also making available a beta release of Pulumi Azure Native Provider 2.0, which decreases the size of the software development kit (SDK) by 60%. This is part of an effort to improve deployment times and reduce resource consumption by, for example, removing support for deprecated application programming interfaces (APIs).

    Finally, Pulumi has added support for property names and values to Resource Search for Pulumi Cloud, a search and analytics tool for cloud computing environments. DevOps teams can now additionally search for instances of, for example, versions of databases or specific availability zone as a result.

    Aaron Kao, vice president of marketing for Pulumi, said previously the company made available a separate tool for converting snippets of Terraform code. For the first time, Pulumi is also making available a converter plugin in the Pulumi engine which allows conversion tools from other IaC tools to be used.

    Review Stacks, meanwhile, makes it simpler for DevOps teams to pick up an unfamiliar codebase, make changes to both application and infrastructure code and share a live environment to review that code.

    Pulumi has been making a case for an alternative to Terraform that centralizes the management and provisioning of infrastructure using a set of embedded guardrails. One of the major issues enterprise IT organizations are encountering these days is the number of misconfigurations created when developers use tools like Terraform to provision infrastructure on their own. Those misconfigurations often lead to, for example, ports being left open and cybercriminals exfiltrating data.

    That approach is gaining traction among enterprise customers that are looking to centralize the management of DevOps processes, said Kao.

    In the wake of a series of high-profile breaches, more organizations are now reviewing the security of their software supply chains, including how infrastructure is provisioned. Many of those organizations will need to revisit those processes as pending legislation that requires more stringent requirements for building software eventually become law, noted Kao.

    At the same time, many organizations are also looking to improve the productivity of DevOps teams during an economic downturn that limits the number of full-time employees they might be able to hire. The pace at which software is being developed has not slowed, so there is a greater urgency to do more with less.

    It’s not clear how quickly organizations are centralizing the management of DevOps workflows, but with the rise of platform engineering, there is clearly more interest in defining consistent backend processes that multiple DevOps teams are currently managing individually. The challenge is finding a way to achieve that goal without impeding the pace of innovation enabled by embracing DevOps in the first place.

  • US DoJ Makes PyPI Give Up User Data ¦ Tape Storage: Not Dead

    US DoJ Makes PyPI Give Up User Data ¦ Tape Storage: Not Dead

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: PyPI complies with a “string of subpoenas,” and LTO continues to grow, despite predictions of its demise. (more…)

  • Dell Looks to Expand Pool of Available DevOps Expertise

    Dell Looks to Expand Pool of Available DevOps Expertise

    Dell Technologies has quietly launched a training and education initiative to significantly expand the number of IT professionals that have DevOps skills.

    Brad Maltz, senior director for DevOps and developer relations ecosystems for Dell, said the next era of IT will require IT professionals that understand how to automate the management of application development and deployment. Given its long infrastructure history, Dell is especially interested in helping IT teams to use a mix of graphical tools and command line interfaces (CLIs) to programmatically deploy software at much faster rates and at scale, he added.

    Dell has already hired more than 200 specialists to provide that training as part of a DevOps Transformation Services consulting services engagement, said Maltz. The primary area of focus will be working with IT organizations that are embracing platform engineering to centralize the management of DevOps and GitOps workflows, he noted. Many of those teams are now relying on declarative automation tools that ultimately make DevOps more accessible to a wider range of IT professionals, added Maltz.

    The number of IT professionals that employ graphical tools based on IT management frameworks such as ITIL still far outnumber the number of IT professionals that have DevOps skills. The ongoing debate between these two camps essentially comes down to the need to be able to consistently manage IT at scale versus being able to adroitly keep pace with the needs of organizations that are increasingly dependent on software. In the digital transformation era, it’s become clear that more of the organizations that hire IT professionals are placing a higher value on agility, said Maltz.

    Bridging the divide between ITIL and DevOps to address that requirement is largely a psychological challenge that requires IT teams to address a range of cultural issues in addition to mastering different types of tooling, he added.

    Regardless of whether organizations continue to manage IT infrastructure internally or rely more heavily on managed services provided by Dell, the need to continuously deliver secure software faster has become a business issue. The trouble is the level of DevOps expertise within organizations tends to vary widely. A recent survey published by the Continuous Delivery (CD) Foundation found only 22% of respondents are using both continuous integration (CI) and continuous delivery (CD) to automate all building, testing and deployment of code in a production environment. Most organizations that have adopted DevOps make extensive use of CI to build applications, but CD has proven to be more challenging to master. In fact, the CD Foundation survey found only 7% of respondents described themselves as DevOps engineers or specialists.

    The bulk of the managed services that Dell provides are focused on managing infrastructure rather than deploying software, but the company does have a managed DevOps offering based on the application development and deployment platform that an internal Dell DevOps team built to construct and deploy applications. Naturally, Dell has a vested interest in accelerating the rate at which software can be deployed on IT infrastructure it provides and manages.

    It may be a while before the overall size of the DevOps community expands, but the training initiatives such as the one launched by Dell will help. However, it may become a lot easier to find and retain that talent in the months and years ahead. In the meantime, organizations would be well-advised to invest in upskilling their own IT teams. After all, it’s less expensive to train someone that already works within an IT organization than it is to compete against thousands of other organizations willing to pay top dollar for the best DevOps expertise money can buy.

  • Red Hat Unveils Developer Hub to Boost Dev Productivity

    Red Hat Unveils Developer Hub to Boost Dev Productivity

    At its Red Hat Summit conference today, Red Hat unveiled its Red Hat Developer Hub, a developer portal that provides access to an opinionated set of development tools based on the open source Backstage project.

    The portal pulls together a set of curated tools and resources so that developers encounter less friction and avoid the complexity of numerous optional tools that span disparate vendors. The goal is to increase developer productivity while reducing the volume of optional tools and information that must be sifted through to actually deliver production code.

    Red Hat Developer Hub aims to streamline the development process and allow organizations to maximize existing resources and developer skills while reducing sprawl, said Chris Wright, CTO and SVP for global engineering at Red Hat. This will help organizations maintain application development velocity and drive a consistent user experience for developers based on tools and skills they’re already familiar with, Wright told attendees.

    Rather than building and maintaining their own internal developer portal, Red Hat is making a case for using Developer Hub to deliver a platform approach to addressing the complexity and sprawl DevOps teams face. Red Hat Developer Hub is, in essence, an opinionated and optimized version of Backstage that aims to deliver improved developer experiences across environments, including Kubernetes and container application platforms such as Red Hat OpenShift.

    Developer Hub will offer a single pane of glass to view all available developer tools and resources, developer self-service capabilities and security and compliance guardrails for cloud-native application development that comply with enterprise-wide best practices, according to the company.

    Additionally, Red Hat Developer Hub includes a set of ‘golden path’ templates—pre-defined, pre-architected and supported templates for further accelerating application development.

    The developer hub is based on the Cloud Native Computing Foundation (CNCF) incubating open source project Backstage, originally developed by Spotify and donated to the CNCF in 2020. Backstage is now being advanced by contributions from multiple vendors, including Red Hat.

    Backstage has been gaining traction in cloud-native application environments that are especially challenging to develop using multiple tools. There’s a lot more focus on developer productivity, which Backstage addresses by creating a catalog of blueprints that developers can readily consume rather than requiring them to build these capabilities themselves multiple times over. The goal is to create scaffolds that developers can consistently reuse across multiple application development projects.

    Red Hat has also developed Red Hat Plug-ins for Backstage, a bundle of six plug-ins that integrate various key systems into Backstage, helping to extend the project’s functionality, ensure a consistent user experience and improve workflow efficiency. The plugins can be used across all Backstage installations, providing even more flexibility for developers across platforms and Kubernetes distributions and include:

    • Application Topology for Kubernetes, which enables developers to visualize real-time status of application and infrastructure workloads deployed to any Kubernetes target, including Red Hat OpenShift, with greater consistency.
    • Multi-Cluster View with Open Cluster Manager (OCM), which provides a view into clusters from Open Cluster Manager’s MultiClusterHub and MultiCluster Engine in Backstage.
    • Container Image Registry for Quay, which improves the integration and speed of interactions with Quay registries by providing a view into container image details. This includes security vulnerabilities (CVEs) associated with deployed images.
    • Pipelines with Tekton, which gives users a view into the details of all Tekton pipeline runs and their respective statuses across all services.
    • Authentication and Authorization with Keycloak, which enables platform engineers to load users and groups from Keycloak into Backstage, allowing the use of multiple authentication providers to be applied to Backstage entities.
    • GitOps with Argo CD, which helps track the health and status of the Argo CD status for services within Backstage.

    Red Hat said additional plugins are in the works. Red Hat Developer Hub will be generally available later this year, but Red Hat is offering a ‘test drive’ through the upstream IDP project Janus.

     

  • What’s the Difference Between DevOps and Platform Engineering?

    What’s the Difference Between DevOps and Platform Engineering?

    The DevOps space is buzzing with a new trend—platform engineering. Platform engineering has been gaining steam over the last few years but has really peaked in interest lately. In fact, according to a study conducted by Puppet Labs, 51% of organizations have adopted platform engineering or plan to do so in the next year. But what exactly is platform engineering, and how is it different than DevOps?

    Below, we’ll explore the nuances between these two concepts. We’ll review what DevOps and platform engineering are and see how they’re similar and dissimilar. We’ll also outline the benefits of platform engineering and suggest some best practices to make the most of this emerging discipline.

    What is DevOps?

    DevOps is a broad topic, but it generally involves efforts to increase software release efficiency and frequency. Some of the hallmarks of the DevOps discipline include reducing siloes between teams and increasing collaboration. DevOps is often directly associated with various tools that automate the process of moving code from a local environment into production, also known as continuous integration and continuous delivery (CI/CD). Monitoring, incident management and configuration management tools also support the DevOps process.

    “DevOps reflects more of a philosophy and a way of working,” said David Sandilands, senior solutions architect for Puppet. DevOps enables more of a shift left mentality. Instead of having operations teams “guarding the gates,” developers work in partnership with Ops. In doing so, groups can decrease build time significantly, he said. This style helps engineers incrementally and rapidly release new features, replacing old waterfall development cycles.

    The ultimate goal of DevOps is to automate the software life cycle and shorten the feedback loop. Thus, DevOps takes a life cycle approach to shipping applications more efficiently, streamlining phases such as build, release, configure, monitor, plan, create and verify. Microservices architecture and containerization also often play a crucial role in enabling DevOps.

    What is Platform Engineering?

    Platform engineering is all about developing a self-service internal platform that serves the needs of developer consumers. Although DevOps opened things up and empowered individual developers with advanced automation, it left organizations with scattered tooling and processes, especially at large enterprises. For example, various infrastructure-as-code (IaC) configurations might exist for similar functions within the same organization but be unknown to engineers in different departments. Not to mention, overseeing a growing DevOps pipeline means added responsibilities around continual updates and security.

    Platform engineering places more control in the hands of a central platform team rather than individual developers. These teams typically treat the platform as a product. They do so by evaluating the needs of developers across an organization, building out internal self-service tools that suit the needs of most consumers and maintaining the platform.

    The 2023 State of DevOps Report found that this self-service approach typically handles areas like deployment, operation of infrastructure, monitoring production applications and security and compliance. Other capabilities, such as abstracting Kubernetes or overseeing secrets management, may also play a role in these efforts.

    Benefits of Platform Engineering

    • Improve developer experience: Platform engineering can enhance the experience a developer has with their software build and release systems. A better developer experience can do loads to improve morale and create a more productive engineering atmosphere.
    • Standardize DevOps practices: By bringing together common DevOps approaches into a shared platform, organizations can begin to standardize with reusable build processes and automated infrastructure. This can unite an organization on common runtimes, observability methods and deployment processes. This can streamline the introduction of DevOps practices to an organization.
    • Secure the DevOps pipeline: Various vulnerabilities exist within tooling that comprises modern CI/CD chains and cloud-native open source software. By outsourcing DevOps responsibility to a shared platform team, the organization can coalesce around more secure practices and avoid overlooking important updates or holes.
    • Supply additional guardrails: Platform teams might also assist in the acquisition of new tools as well as outlining and enforcing policies for software infrastructure. While this may come at the cost of customization, additional guardrails can help avoid shadow IT and governance around internal development. Letting platform teams handle it could make maintenance easier across an organization.

    Platform Engineering: Under the DevOps Umbrella

    I don’t see platform engineering as replacing DevOps. Instead, I view platform engineering as falling under the umbrella of DevOps since platform engineering takes DevOps processes and makes them more reusable and self-service. Both use a lot of automation and share many of the same end goals.

    However, there are nuances between the two approaches that reflect the recent trends in the IT market. With DevOps, the market was more interested in shifting responsibility left and empowering individual developers to oversee the entire software life cycle. Now, as sprawl and shadow IT emerge, the pendulum seems to be shifting back toward a bit more centralization and governance with platform engineering.

    Platform engineering is typically more of an in-house development initiative. So, what are some best practices IT divisions should remember to get the most out of platform engineering? Hope Lynch, senior director of platform at Cloudbees, shared some great strategies to consider that revolve around treating the platform as a product. Doing this involves accepting feedback from developers, not controlling everything and always maintaining a focus on developer experience.

    Hopefully, by following these tips, organizations should have a better grasp of the emerging discipline of platform engineering and have the confidence to begin introducing platform-based concepts within their own environments.

  • Spotify Adds More Plugins for Backstage Developer Portal

    Spotify Adds More Plugins for Backstage Developer Portal

    Spotify this week added additional plugins for its open source Backstage platform that is used to build developer portals. The new plugins make it simpler to address role-based access and access Insights, a tool from Spotify that tracks Backstage usage trends.

    In addition, Spotify is also enhancing a Soundcheck plugin for Backstage that is used to visualize and track development of software components. Forthcoming capabilities include a no-code interface that makes it possible to programmatically create checks of code without writing any code.

    DevOps teams will also be able to view, export and understand how teams and components are doing compared to established best practices and monitor trends, graphs and historical views, and receive notifications when levels change.

    Finally, Spotify is committing to adding additional integrations with third-party tools and platforms such as Snyk, Sonarqube and the open source Argo continuous delivery (CD) platform.

    There are now five plugins available via a Spotify Plugins for Backstage subscription service, and Spotify said more are planned.

    Meg Watson, a group product manager at Spotify, said Backstage has been gaining traction in cloud-native application environments that are especially challenging to develop using multiple tools. Because of Backstage, developers are not only more productive but turnover has also been reduced because developers are provided with a better experience, she noted.

    There’s a lot more focus than ever on developer productivity, which Backstage addresses by creating a catalog of blueprints that developers can readily consume rather than requiring them to build these capabilities themselves multiple times over. The goal is to create scaffolds that developers can consistently reuse across multiple application development projects.

    Backstage was donated to the Cloud Native Computing Foundation (CNCF) and is now being advanced by contributions from multiple vendors. Much of that focus is on lowering the barrier of adoption for the platform, said Watson. Specifically, Spotify is working on a QuickStart for Backstage edition of the platform that is simpler to install, she said.

    In general, Spotify is trying to strike a balance between centralizing the management of DevOps workflows and the need to enable developers to define workflows that are natural to them versus ones that have been imposed on them, added Watson.

    It’s not clear to what degree Backstage will help fuel the adoption of a shift toward platform engineering to centralize the management of DevOps tools and platforms. The concept of a portal through which developers can self-service their own needs may not be new, but as an open source project, Backstage has made it simpler for many organizations to achieve that goal.

    Just about every DevOps team is now being asked to find ways to help improve developer productivity at a time when many organizations are trying to do more with fewer resources. Regardless of the motivation, however, it’s apparent that DevOps workflows are continuing to evolve and mature.

    In the meantime, DevOps teams would be well-advised to eliminate as many bottlenecks as possible before they negatively impact developer productivity.