devopsy.kinsta.cloud

Tag: infrastructure as code

  • Microsoft Adds Slew of Developer Tools to Azure

    Microsoft Adds Slew of Developer Tools to Azure

    At its Build 2023 conference this week, Microsoft announced general availability of an Azure Deployment Environment service that provides developers with a portal through which they can employ infrastructure-as-a-code (IaC) templates based on Terraform or Azure Resource Management files.

    In addition, Microsoft has enhanced Microsoft Dev Box, a workstation environment for developers that run Azure, to include a portal through which they can manage multiple environments.

    Scheduled to be generally available in July, Microsoft Dev Box is also previewing a capability that makes it possible to customize environments using configuration-as-code files stored in a Git repository. Microsoft is also previewing tighter integration between Dev Box and its Visual Studio integrated development environment (IDE) and Windows to further customize developers’ environments.

    Scott Guthrie, executive vice president of the cloud and artificial intelligence (AI) group at Microsoft, told conference attendees these and other tools for building cloud-native applications, along with GitHub Actions, make it simpler than ever to build applications in an Azure cloud service that includes an integrated continuous integration/continuous delivery (CI/CD) platform.

    Microsoft is also previewing GitHub Advanced Security for Azure DevOps that natively embeds automated security checks using scanning tools into the Azure DevOps platform.

    Other additions to the Azure environment include tools for centralizing the management of application programming interfaces (APIs) using an Azure API Center service that is available in preview.

    The Azure Event Grid has also been updated to enable it to use HTTP to pull delivery of discrete events in addition to pushing them.

    Finally, Microsoft is also previewing generative artificial intelligence (AI) capabilities to make it simpler to manage cloud costs by, for example, using prompts to summarize invoices or breakdown costs.

    Microsoft, along with its GitHub subsidiary, is at the forefront of applying generative AI to application development using tools like GitHub Copilot that developers can use to write better code faster. That copilot capability is now being disseminated across the portfolio of applications and services that Microsoft provides.

    It’s not yet clear what the downstream impact of such tools are just yet, but the overall quality of the applications being built should improve as, for example, fewer mistakes are made that can lead to vulnerabilities.

    Obviously, the way application software is built, constructed and secured is about to change in the generative AI era. Thanks to its alliance with OpenAI, Microsoft has a significant head start on rivals building generative AI platforms that can similarly be applied to improve developer productivity.

    In the meantime, DevOps teams should assume the volume of code simultaneously moving through pipelines is about to dramatically increase as developers become more productive. There will inevitably be more opportunities to apply multiple forms of AI to reduce any DevOps bottlenecks that might arise, but it’s clear most of the immediate benefits are being provided to developers.

    One way or another, however, the overall rate at which applications are built and deployed is about to change drastically. The issue is understanding how those advances will impact downstream DevOps workflows that were constructed for a different era of application development.

  • Infrastructure-as-Code Pitfalls in Platform Engineering

    Infrastructure-as-Code Pitfalls in Platform Engineering

    In the context of cloud operations, the last decade was ruled by DevOps and infrastructure-as-code (IaC). But what is true DevOps? Is it developers running their own operations? Is it operators learning development skills to automate infrastructure provisioning using infrastructure-as-code (IaC) tools like Terraform?

    Though DevOps is widely understood to incorporate both of these, software developers are far from fully understanding or operating infrastructure. IaC is purely a DevOps tool. When an organization claims to have automated its infrastructure, they mean automation for the efficiency of the DevOps teams, not automation for developers. Developers are still left waiting on DevOps for days for even small infrastructure changes.

    While there were efficiencies gained by the current DevOps model, there is an increasing acknowledgment that we require a better approach to enable developer self-service. This has led to the platform engineering discipline.

    The high-level goals of platform engineering are:

    • Developer self-service for significant parts of infrastructure updates without DevOps subject matter expertise.
    • Built-in security and compliance controls.

    Infrastructure-as-Code (IaC) Limitations in Platform Engineering

    A DuploCloud survey of 40+ enterprises that have made substantial investments in platform engineering showed that the prevalent approach is for DevOps teams to build a DevOps platform with IaC as a core underlying technology. They create templates for the organization’s use cases and publish them in a CI/CD pipeline or a self-service catalog.

    Here are the top reasons why platforms that build on top of IaC are failing these platform engineering goals:

    1. IaC templates are rigid and not flexible enough to meet changing developer requirements: It is true that DevOps teams can anticipate cloud infrastructure topologies to some extent and have IaC templates for those with a few customizable parameters. But in a microservices world, there are thousands of other workflows and topologies possible based on changing application needs and security controls. Large data centers and edge infrastructure make this even more complex. A manual approach, relying solely on DevOps personnel to constantly build and update myriad combinations in static scripts simply can’t scale.
    2. Scripting tools can’t build life cycle management: In cloud operations, people-triggered changes are only a subset of possible use cases. Many asynchronous operations need to be continuously performed. These range from detection of configurations, from desired state to reverting, or complex configuration scenarios where individual components have to be set up asynchronously and brought together later. It could be as simple as a certain component going down and needing to be restored. IaC is a script that runs when triggered to completion. It has no active life cycle to operate continuously in the background.
    3. Inability to build a concept of an environment: In any orchestration system, users have a concept of the environment they want to build. When they log in to the platform and navigate to their environment, they expect to update as well as view the state and aspects of resources in that environment, be it the provisioning status, metrics, logs, faults, audit logs, compliance posture, etc. They may want to perform debugging functions, such as restarting services, SSHinginto a VM or accessing a resource’s cloud console (S3 Console, for example) using access control boundaries within that environment. For example, in Kubernetes, you choose the namespace to be the environment and management software like Rancher on top of K8S to provide these functions. If we must replicate this same concept across a broad infrastructure-level platform, we cannot do it using only IaC. We need something with a scope that spans all cloud operations, like Kubernetes. Terraform is a configuration updating and management software, not an orchestration system.

    Many platform teams have tried to work around these problems with point solutions by using a disparate set of jobs for certain aspects of life cycle management, building a thin UI shim on top of cloud accounts for visualization of resources while redirecting to other systems like Datadog for logging, metrics and alerts. But for most of the use cases, the DevOps team is still very much in the operations workflow. This completely defeats the concept of developer self-service as well as continuous compliance goals.

    Learning From Other Successful Platforms

    Two recent examples of successful cloud platforms are Amazon Web Services (AWS) in the context of infrastructure-as-a-service (IaaS) and Kubernetes for container orchestration. These are distributed system implementations using higher-level programming languages like Java and Go. You can’t build such complex systems using scripts and jobs.

    Building a true DevOps orchestration platform requires a systems design approach. It also takes expert systems engineers and many years to build and mature, like it did for K8s and IaaS in the Public Cloud.

    Conclusion

    Since platform engineering teams need systems design expertise that is likely outside of the scope of the typical DevOps role, we need the software development function to intervene with distributed systems programmers and many years of investment. Most organizations trying to build in-house platforms don’t have or don’t want to make this type of investment, as it may not be their core business. Megascale tech companies like Facebook, Uber, Netflix, etc. will likely build an in-house solution, as they have the talent and scale, but for the masses, it is likely that the solution will be an off-the-shelf product from an ISV.

    The industry has solved IaaS with public cloud, container orchestration with Kubernetes and observability with platforms like Datadog. The platform engineering category is heading in the same direction.

  • Implementing SLOs-as-Code: A Case Study

    Implementing SLOs-as-Code: A Case Study

    This article was co-authored by Stephan Lips, Devin Cunningham and Justin Hoang of Procore and is a preview of their talk for SLOconf 2023 on May 15-18, 2023. To watch this talk and many more like it, register for free at sloconf.com.

    By managing service level objectives (SLOs) as code, we can co-locate SLO definitions and ownership with the product code and team. This supports horizontal scaling of SLO ownership while establishing a single source of truth and adding transparency, integrating with the code management process, and creating an audit trail for SLOs.

    When you measure the reliability of hundreds or even thousands of products and services across an enterprise, ownership of SLOs should not reside with a single team. There are a few aspects that make scaling SLO ownership horizontally across teams much more efficient: A standardized SLO management platform, process architecture and automation across the enterprise. Managing SLOs-as-code alongside product code is a step towards 360° product ownership and enables us to automate SLO updates via continuous integration.

    Objectives and Architecture

    The larger objective behind the SLOs-as-code approach is to move towards 360° product ownership. At Procore, teams own their product’s code and processes, including aspects like testing, performance, reliability, automated deployment pipelines and deployment configurations. Adding SLO ownership to a team’s portfolio will also help improve product reliability.

    Implementing SLOs-as-code involves three architectural components, which will be discussed in detail in their own respective sections. In summary, these components are:

    • SLO objects and definitions: Standardized SLOs and related resources facilitate centralized management and automation.
    • Automation: SLOs as code are our single and reproducible source of truth, which we automate via Continuous Integration.
    • A common platform: Nobl9 is the SLO management platform upon which SLOs, related resources, and automation are built. It aggregates the various data streams that power SLIs.

    SLO Objects and Definitions

    Procore’s Observability team has designed our SLO-as-code approach to scale with Procore’s growing number of teams and services. Choosing YAML as the source of truth allows Procore a scalable approach for the company through centralized automation. Following the examples put forth by openslo.com and embracing a ubiquitous language like YAML helps avoid adding the complexities of Terraform for development teams and is easier to embed in every team’s directories. We used a GitOps approach to infrastructure-as-code (IaC) to create and maintain our Nobl9 resources.

    The Nobl9 resources can be defined as YAML configuration (config) files. In particular, one can declaratively define a resource’s properties (in the config file) and have a tool read and process that into a live and running resource. It’s important to draw a distinction between the resource and its configuration, as we’ll be discussing both throughout this article. All resources, from projects (the primary grouping of resources in Nobl9) to SLOs, can be defined through YAML.

    Figure 2. A simple visualization of Nobl9 object relationships. A Project is a top-level object. Projects contain Services, SLOs are attached to Services, and SLOs can trigger alerts via project-scoped Alert Policies. Finally, Role Bindings define who has access to the Project contents.

    Procore adopted a hybrid approach to organizing our Nobl9 configuration so that the observability team can review systemic changes while teams still own changes to their service SLOs. A separate central repository is the source of truth for all other Nobl9 configurations, such as Project and Alert Policy definitions. As projects have a one-to-many relationship with their services, it would quickly become a game of “guess where” if the project configuration were defined within one of its service repositories. The central repository is owned by the observability team and allows us to manage permissions through pull request reviews submitted by product teams for their non-SLO Nobl9 resources. Once requests are merged, our automation applies the changes. The SLO definitions are co-located with the service’s code. Teams self-regulate these resources to maintain their SLOs internally.

    Automation

    Automating the creation and modification of Nobl9 resources is essential to Procore — it makes iterating on the deployment process quick and painless for our engineering teams. Automation removes human error and possible complications that usually come along with manually applying config files via CLI tools.

    With that in mind, the observability team created a CI job/workflow that our engineering teams copy into their project repo during their SLOs-as-code onboarding. Procore’s CI job uses the Nobl9 sloctl docker image, so we do not have to install the sloctl CLI tool in our CI containers. We configured this job to only apply the configs that have been added or updated, which helps to future-proof our pipeline as we scale the number of our SLO configs.

    Figure 3: SLOs-as-code Automation Workflow

    This requires the following steps in the workflow:

    1. An engineer creates or modifies a config and commits the changes to GitHub.
    2. The engineer opens a PR, gets reviews and merges their changes into the main branch.
    3. CI picks up the changes in GitHub and kicks off the following steps:
      1. Using git diff and regex pattern matching, we figure out what configs have been added or changed from the most recent merge to main:

    #!/bin/bash
    ADDED_OR_CHANGED_CONFIGS=$(git diff –name-only –diff-filter=d HEAD^ HEAD | { grep -E “^${NOBL9_RESOURCE_DIR}\/.*\.y[a]?ml$” || :; })

    2. We loop over the added/changed configs and run sloctl apply using the nobl9/sloctl docker image, which updates our resources in Nobl9:

    #!/bin/bash
    for file in “$ADDED_OR_CHANGED_CONFIGS”
    do
      docker run nobl9/sloctl:v0.0.80 apply -f “$file”
    done

    3. We use the same workflow in our centralized IaC repo that manages our projects, services, role-binding and alert policy configs.

    Common Platform

    Our platform requires another key feature for SLOs-as-code to work: We need to be able to consolidate different observability data sources and telemetry streams that power SLIs. This platform needs to support standardized data formats to define SLOs and related entities, such as alert policies for different conditions and trends, while allowing for the management of these formats via automation. We evaluated three vendors that offered SLO solutions, and only Nobl9 met all our requirements, which extended beyond the SLOs-as-code topic of this article.

    Observations

    Adopting SLOs-as-code can be impeded by three particular barriers of entry: New tooling (Nobl9 platform), data and format (SLO configs and resources) and process. To drive adoption, we have designed a self-service model supported by an open-door consulting practice. While this is an ongoing learning process for all involved, we are seeing early signs of success of an approach that uses:

    • Detailed, step-by-step instructions, including code snippets and templates.
    • Screencast tutorials supplementing the instructions.
    • Initial face-to-face, white-glove meetings to explain concepts and provide demos. The personal experience and conversation in our experience created considerable enthusiasm and goodwill among our teams. It proved to be a much better approach than just pointing them to documentation.
    • Fast feedback cycles with onboarding teams to improve instructions.

    Quite frequently, teams stated their primary challenges were not the technical aspects of SLOs-as-code discussed in this article, but how to design and improve on meaningful SLOs. We addressed this by authoring articles like Black Box SLIs and other internal resources for guidance. While posts can be a useful tool, we found that working with teams and discussing how their product impacts the user experience was more beneficial to modeling their SLOs using observability data.

    Last but not least, we are integrating with our embedded SRE (ESRE) team to help onboard more services. When a product team engages with ESRE on their deployment strategy and configuration, ESRE also discusses onboarding to SLOs-as-code and basic SLOs, such as error rate or duration.

  • Pulumi Brings Advanced Analytics and AI to IaC Platform

    Pulumi Brings Advanced Analytics and AI to IaC Platform

    Pulumi today launched a platform that leverages analytics and artificial intelligence (AI) to generate infrastructure-as-code (IaC) from natural language prompts in addition to automating infrastructure management.

    Pulumi CEO Joe Duffy said Pulumi Insights will substantially improve the productivity of DevOps teams in addition to ultimately reducing the total cost of cloud computing.

    At the core of Pulumi Insights is an analytics engine automation application programming interface (API) that enables DevOps teams to launch queries using a search interface or natural language prompts across more than 100 cloud computing platforms. Results can then be organized to highlight the most-used clouds and resources broken down by project and environment using built-in dashboards. There’s also a REST API that makes it possible to export that data to a data warehouse such as Snowflake, Amazon Redshift, Google BigQuery and Azure Synapse.

    In addition, Pulumi has created a companion website and command line tool to make it possible to create code for provisioning infrastructure. That uses large language models like those at the core of generative AI platforms such as GitHub CoPilot or ChatGPT that promise to eliminate much of the toil associated with writing code.

    Pulumi has been making a case for an alternative approach to IaC tools, such as Terraform, that gives enterprise IT organizations more control over how IT environments are provisioned. Too many developers with limited cybersecurity expertise are provisioning infrastructure themselves, resulting in misconfigurations that cybercriminals can then easily exploit.

    The Pulumi approach also makes it simpler for IT teams to create platform engineering teams that can centrally apply guardrails that prevent those issues from arising without slowing down the rate at which applications are being built and deployed, noted Duffy.

    There’s also a much greater focus on improving the productivity of DevOps teams during uncertain economic times as organizations look to either limit or reduce the number of full-time employees they hire or retain.

    For one reason or another, DevOps teams are going to be required to find ways to become more productive as the number of applications being built and deployed continues to increase. AI clearly has a major role to play in enabling organizations to achieve that goal. However, before organizations can reliably benefit from those advancements, they will need to ensure the accuracy of the data those AI models use to surface recommendations and generate code. Otherwise, no matter how advanced these platforms are, the results will be suboptimal.

    In the meantime, IT organizations will need to revisit how IaC tools are being employed in an era where the focus on securing software supply chains has dramatically increased after some high-profile breaches. The challenge is that, in addition to all the applications currently being developed, there are thousands more that were deployed by developers without the same emphasis on cybersecurity.

  • env0 Extends Workflow Platform for Provisioning Infrastructure

    env0 Extends Workflow Platform for Provisioning Infrastructure

    env0 today announced it has added support for additional infrastructure-as-code (IaC) tools and the Microsoft Azure DevOps platform to its workflow automation and management platform.

    Fresh from raising an additional $35 million in funding, env0 CEO Ohad Maislish said while Terraform remains the most widely used IaC tool, there are now more organizations using alternatives such as CloudFormation from Amazon Web Services (AWS) along with Pulumi and Terragrunt.

    Other capabilities added to the env0 platform include automated drift detection, multi-tier workflows, auditing, external logging platform integration, support for plug-ins and expanded role-based access controls and OpenID Connect (OIDC) authentication.

    Finally, env0 is extending its existing Terraform support to include support for remote backend, module registry, remote plan, and the ability to plan and apply pull requests. The capabilities are especially critical for organizations that are now building microservices-based applications that require more sophisticated workflows to provision infrastructure resources, noted Maislish.

    Most IaC tools are designed to enable developers to address technical implementation and provisioning issues without regard for user experience, security and governance, or business goals. The env0 platform makes it possible to create infrastructure environments with guardrails in place that enable DevOps teams to centrally address governance issues, noted Maislish.

    In general, there’s now a lot more focus on continuous delivery (CD) as organizations look to continuously update applications, said Maislish. That requires a more structured approach to provisioning infrastructure across teams of developers that is managed by a platform engineering team, he added. The challenge is that CD is dangerous—in the sense that mistakes can easily be made, noted Maislish.

    It’s not clear whether organizations are embracing platform engineering to centralize the management of DevOps workflows, but the number of organizations that have mastered CD best practices remains low. Most organizations are adept at continuous integration (CI), but given the number of unique platforms on which applications are deployed, CD has proven more challenging to adopt. In theory, a platform engineering team should make it easier for DevOps teams to define a consistent approach to CD that could be adopted across an organization.

    Regardless of the DevOps approach, the overall goal is to improve developer productivity. The problem is that in the absence of any guardrails, developers have been provisioning cloud infrastructure as best they can. Not surprisingly, cloud computing environments are rife with misconfigurations that create vulnerabilities that cybercriminals have become more adept at exploiting. The fundamental issue is not such much the cloud infrastructure that is available as much as it is the processes employed to provision the resources. In their rush to build and deploy applications, developers will sometimes ignore best practices unless guardrails force the issue.

    One way or another, IT organizations are exercising more control over cloud computing environments as the security of software supply chains becomes a bigger issue. The only thing that remains to be resolved now is how to put much-needed guardrails in place in a way that adds the least amount of friction possible.

  • Quali Update Strengthens Infrastructure Security

    Quali Update Strengthens Infrastructure Security

    Quali today announced enhancements to its Torque automated infrastructure platform to add support for security scans and Open Policy Agent (OPA) software being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    OPA enables organizations to implement cybersecurity policies as code. The challenge many organizations have encountered is that not many developers have mastered the programming language required to implement OPA. Quali is embedding OPA in a platform that enables IT teams to implement it using a graphical tool that enables central management and application of those policies.

    David Ben Shabat, vice president of research and development at Quali, said the overall goal is to implement scans and policies in a way that reduces the number of misconfigurations that are made when organizations provision infrastructure-as-code (IaC).

    As part of that effort, Torque now flags any policies that fail validation. In addition, administrators can pick and choose which OPA policies to import, including both out-of-the-box and existing custom policies. Those policies then enable Torque to automate approvals or denials of deployments based on compliance with those policies after scans and other health checks have been run.

    Quali has been making a case for policy-based triggers for cost controls, automated start-up, shut down and pausing of IT environments to give IT teams more control over infrastructure using a software-as-a-service (SaaS) platform. That approach enables IT teams to provide a set of guardrails for provisioning infrastructure that prevent mistakes from being made, noted Ben Shabat. Those controls can be applied using the graphical tool Quali provides or programmatically via an application programming interface (API) that Quali exposes, noted Ben Shabat.

    It’s not uncommon for a port to be left open, for example, which cybercriminals later exploit to exfiltrate data. Quali is offering a DevOps platform that allows for central application of policies that act as guardrails to prevent developers from making these types of mistakes as an alternative to scripts created using tools such as Terraform. That approach also provides more visibility into how infrastructure resources are being consumed.

    Every minute a developer spends managing infrastructure is one less minute they could be spending writing code. As application environments become more complex, it is becoming more challenging for developers to manage infrastructure-as-code. The issue is that DevOps teams who could provision infrastructure for developers are overwhelmed by the number of projects they need to manage. As a result, DevOps teams are becoming bottlenecks. Naturally, that only encourages more developers to provision infrastructure on their own, leading to more mistakes.

    It’s not clear whether IT organizations will look for an alternative to provision infrastructure in a way that doesn’t require developers to have as much infrastructure expertise. However, as organizations look to lock down software supply chains, the way infrastructure is provisioned using code will, in many cases, need to be reconsidered within the context of a larger DevSecOps workflow.

  • The Differences Between DevOps and GitOps

    The Differences Between DevOps and GitOps

    The transition to a DevOps model has been welcomed by many organizations that are looking to implement more efficient development processes. However, there are multiple models to choose from, with GitOps a popular choice for many. So what is the difference, and which one is better?

    In this article, we will discuss what each model entails and their key differences and establish if one is better than the other for your business.

    What is DevOps? – An Overview

    The DevOps model involves several tools and practices designed to make software development much more efficient. This increase in efficiency is achieved by implementing more automation, adopting continuous delivery practices and improving the communication channels between the developers and operations. 

    Ultimately, DevOps is designed to assist organizations in deploying new products and systems faster. By establishing more collaboration between development and operational teams, changes can be made quickly to best suit the business’s overall goals. 

    There is no single tool that is required for DevOps. In fact, your team can use any tool that effectively improves communication and automates key tasks. 

    How to Integrate a DevOps Strategy

    Adopting and integrating a DevOps strategy into your organization’s overall workflow is not a difficult process, and it can be achieved by following these steps: 

    1. Create a continuous integration/continuous delivery (CI/CD) pipeline.
    2. Configure this new pipeline to enable the automated deployment of changes to your development/production environment.
    3. Set up an alert and monitoring system for your development environment. 
    4. Create collaboration channels so operational teams can inform the developers of new opportunities to improve the business’s workflow. 

    An Overview of GitOps

    GitOps is a modern concept that was initially introduced by the Kubernetes management system Weaveworks. This model uses Git, an open source, distributed version control system, to track any changes to a system and who has made them. 

    The GitOps model uses a Git repository as the single source of truth (SSoT), improving collaboration between developers and operational team members. This, in turn, allows organizations to automate the deployment process. 

    By leveraging CI/CD pipelines like DevOps, changes are automatically applied to the codebase without any need for manual input. Thanks to Git, these changes can also be easily reversed if needed, reverting to a previous working version. 

    How to Integrate a GitOps Strategy

    GitOps can be integrated into a business’s workflow quite simply by following four key steps: 

    1. Set up a Git repository for your organization’s codebase. When integrating a GitOps strategy, an organization does not need to use GitHub exclusively, as any code repository that uses Git commands can do the job. 
    2. Connect this repository to a CI/CD tool.
    3. Configure your chosen tool to deploy automatic changes to your development environment. Popular tools include Jenkins and Circle CI. 
    4. Set up monitoring and alerts for your development environment. 

    It should be noted that the CI part of GitOps still needs to go through your DevOps tool, whether it’s GitHub Actions or AWS CodeBuild. 

    The Differences Between DevOps and GitOps

    The core difference between the two is that GitOps allows teams within an organization to manage and deploy changes to infrastructure using Git as the single source of truth. DevOps, on the other hand, focuses on effectively connecting development and operational teams using a tool of the organization’s choosing. 

    Either model can offer a number of benefits in terms of development and deployment, but there are notable differences between the two that should be considered. It should also be noted that DevOps and GitOps can work in unison as long as continuous integration (CI) is delivered using your chosen DevOps tool.

    DevOps – Key Elements

    • The DevOps model is considered a working culture.
    • DevOps can use a variety of CI/CD pipelines and is not restricted to any one tool.
    • This model increases automation and the regularity of deployment, reducing any human input. 
    • Cloud configuration can be used for coding, supply chain management and other key requirements.
    • The pipeline separates development and operations processes while improving collaboration between the two.

    GitOps – Key Elements

    • GitOps is a development and deployment technique that relies on the Git version control system.
    • Git is used as the single source of truth for all changes to the codebase.
    • GitOps can use Kubernetes and infrastructure-as-code (IaC) practices and various other CI/CD pipelines. 
    • The GitOps model is designed to increase development times and reduce reliance on lengthy scripts.
    • It also enables a smoother transition between a project’s development and operation stages.

    DevOps Vs. GitOps – Which Model Is Better?

    Both GitOps and DevOps are becoming increasingly popular when it comes to adopting new approaches to improve software development processes and making deployment quicker and more efficient. As outlined above, the difference between the two models largely comes down to the tools that are used, but the result is roughly the same in terms of the overall benefits. 

    Deciding which approach is the right one for your organization or determining which one is “better” will always come down to the needs and requirements of your organization. In fact, the ideal scenario would be to integrate GitOps so it works alongside your DevOps approach. 

    GitOps is not a replacement for DevOps, as the CI process still relies on DevOps tools. However, GitOps can improve the effectiveness of the DevOps team thanks to continuous delivery processes. Using Kubernetes-native mechanisms and commands and sticking to standard source code principles, GitOps does not require developers to change how they work, guaranteeing a seamless implementation process.

    Wrapping Up

    DevOps is a working culture that improves collaboration between development and operations teams and utilizes automation tools, resulting in rapid and more efficient software development. GitOps, on the other hand, uses Git as the single source of truth, making it easy to track changes and further increase development times thanks to continuous delivery processes. 

    Although each model can be used exclusively, they can also work alongside each other to deliver rapid software development and high levels of collaboration. Therefore, it is unfair to claim that one model is better than the other, as they can both play their part in an overall DevOps strategy. 

  • Devs Increasingly Rely on IaC to Manage Multiple Clouds

    Devs Increasingly Rely on IaC to Manage Multiple Clouds

    A survey of more than 300 DevOps professionals conducted by Firefly, a provider of a platform for automating cloud provisioning, found that more than a third of organizations (36%) have allocated the equivalent of two or more full-time engineers per month to writing infrastructure-as-code (IaC) templates.

    A full 82% are also deploying the templates within the context of a DevOps pipeline, with 68% reporting more than half of their cloud instances are now managed using IaC tools, according to the survey. A total of 27% reported they are now managing every aspect of IT as code.

    The challenge is that 61% of respondents also admitted they can’t automatically detect configuration drift over time, with more than a third (35%) spending days or weeks remediating IaC templates. Well over half of respondents (56%) said they are also retrofitting existing cloud instances using IaC tools.

    Cindy Blake, vice president of marketing for Firefly, said this issue is likely to become more challenging to address as the number of clouds in use continues to increase. The survey found 44% of respondents already use multiple cloud providers. Overall, 42% reported having more than 10 cloud accounts.

    The top IaC benefits identified by survey respondents are improved efficiency (75%), increased reliability (55%) and easier compliance (52%) and simpler troubleshooting (49%).

    Not surprisingly, the survey identified knowledge (46%) followed closely by complexity (43%) as the top challenges organizations face when using code to manage cloud infrastructure. Factors increasing complexity included cost and complexity (70%), followed by security (61%).

    In addition, only 14% of respondents reported that all their engineers are IaC literate. More than a third (36%) said they only have one IaC-literate engineer. As a result, many organizations are now attempting to navigate IaC bottleneck issues that could be alleviated if they relied more on automation that can be integrated within a continuous integration/continuous delivery (CI/CD) platform, noted Blake. She added that many organizations are already allocating too much engineering time to manually writing IaC templates.

    Those manual efforts often result in misconfigurations that are easily exploited by cybercriminals, added Blake. Many developers using IaC tools such as Terraform have had little to no cybersecurity training, so the odds they will make a mistake are high.

    At a time when more organizations are focused on the productivity of DevOps teams and ensuring the integrity of software supply chains, there is a clear need to automate processes that increasingly span multiple clouds. More organizations than ever are employing multiple clouds to ensure application availability in the wake of a series of outages over the past few years, Blake said.

    Regardless of the motivation, multi-cloud computing is here to stay. Developers opt for different clouds based on the performance capabilities they can provide for various classes of workloads. There may be one cloud platform that runs the bulk of the workloads, but the number of organizations that have standardized on a single cloud are comparatively few. The issue is finding the best way to manage multiple clouds that doesn’t require as much manual effort so organizations can reduce the total cost of managing IT environments.

  • Best of 2022: Environments-as-a-Service: Free Your Devs

    Best of 2022: Environments-as-a-Service: Free Your Devs

    As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    Environments are the bane of a DevOps engineer’s existence and have been since the invention of the server. They can create giant bottlenecks that hamper productivity and suck the life and motivation out of developers. But they’re also completely necessary.

    DevOps teams spend a ton of time building and maintaining their development environments. They are critical to testing new websites, apps, products and updates; making sure everything works the way it is supposed to. And not having enough environments can slow down workflows as developers wait their turn to use an environment.

    There is a new push in the software industry to provide environments-as-a-service (EaaS), removing the annoying yet necessary work of building and maintaining environments and giving DevOps engineers more freedom to tackle more challenging, revenue-generating projects.

    EaaS enables companies to use a third-party service to create a new development environment on-demand, including data, infrastructure and code—all the things necessary to build and run an app. Environments-as-a-service bring major improvements to businesses, saving money and time while increasing the velocity, delivery and deployment speed.

    EaaS offers automation and codification of all the things that would otherwise waste DevOps and infrastructure folks’ time. They should be focused on higher-value tasks, like creating and refining what makes the company or product unique.

    Removing Bottlenecks

    When prepping for a new product or service, teams have to test all associated code to make sure it is functional, secure and that there are no hiccups. Multiple environments are crucial to this process. However, in many cases, teams work within a single, shared staging environment.

    This setup means developers have to wait in line to test their part of the product. These bottlenecks are among the biggest causes of delayed releases.

    Time is money, and in this case, lost time costs money. We estimated the total cost of managing environments exceeded $45 billion annually, and release delays cost organizations millions of dollars every year.

    Organizations can layer EaaS on top of their infrastructure-as-code (IaC) frameworks, agile workflows and other important software. By doing this, developers can create a new environment with a single click.

    That means developers can make a new environment for every feature branch pull request. EaaS also unlocks customization capabilities, as well. For example, testing requires larger and faster environments that integrate with other key services, while more scaled-down environments work well for processes like demos.

    Now, developers are free to test and develop software, without waiting or stepping on anyone else’s toes. Plus, on-demand environments can also be destroyed when they are no longer needed. That’s capacity planning in real-time.

    Save Time and Money

    Capacity knowledge and better capacity planning mean money saved. Knowing how many environments are needed, and what types, helps organizations avoid over- or under-spending on cloud infrastructure.

    Building out environments can take more than a year for even sizable DevOps teams and cost hundreds of thousands of dollars, not to mention the challenges of ongoing management, integration of new tech and management of cloud resources and costs. A third-party EaaS provider takes care of all of that, freeing DevOps engineers to work on hard problems, rather than maintenance and repeating work that has already been done as technology advances.

    Is Your Organization EaaS Ready?

    Any time a company automates something as important as staging and testing environments, though, it’s important to have all their ducks in a row. Luckily, the best ways to do that match up pretty well with best practices for building out any infrastructure platform:

    ● Using infrastructure-as-code automates the provisioning of infrastructure instead of doing it manually.

    ● Containerizing custom code makes it easier to quickly reproduce it for new environments.

    ● Finally, carefully think about how your environments will behave when they are created dynamically. Simple things like creating environment variables for hostnames and trying not to hardcode variables that are environment-specific are a must.

    Organizations that have followed these steps are EaaS-ready. By automating these painstaking tasks, companies can unleash their DevOps teams to be superheroes for the organization, solving the biggest problems and letting tech take care of the rest.

  • Fiberplane Notebook Tool Fosters DevOps Collaboration

    Fiberplane Notebook Tool Fosters DevOps Collaboration

    Fiberplane today announced it is making available a public beta of a namesake real-time collaboration notebook specifically designed for DevOps teams that need to quickly debug IT infrastructure.

    Fiberplane CEO Micha Hernandez van Leuffen said in the wake of the COVID-19 pandemic, it’s clear that the structure of DevOps teams has permanently changed now that more members of those teams work remotely. The challenge is that the tools provided to DevOps teams to manage IT environments were not designed to foster the level of collaboration that is now required, he added.

    Dashboards that were designed in a different era need to be replaced with tools that make it simpler for DevOps teams to collaboratively debug infrastructure in real-time, noted Hernandez van Leuffen.

    The Fiberplane notebook achieves that goal by using the same operational transformation algorithm used in Google Docs to enable teams to easily collaborate around a document or spreadsheet. The core platform itself is built in Rust and WebAssembly (Wasm), which Hernandez van Leuffen said makes it simpler to integrate Fiberplane as a plug-in into an existing DevOps workflow. Inside a Fiberplane Notebook, DevOps teams can, for example, write and run PromQL queries and display Prometheus charts alongside other data presented via an integrated graph.

    Fiberplane Notebook’s initial integrations are for Grafana Loki, Elasticsearch and Prometheus monitoring tools. It also exposes both an application programming interface (API) and command line interface (CLI) to automate notebook creation based on specific external events, such as monitoring alerts.

    In effect, Fiberplane is taking the concept of notebooks used for collaboration by data science teams and applying it to DevOps workflows, said Hernandez van Leuffen. The goal is to improve DevOps teams’ productivity by reducing the amount of time it takes to resolve any issue, he added.

    As IT environments become more distributed in the age of the cloud and microservices, troubleshooting issues invariably spans the responsibilities of multiple members of an IT team. Fiberplane, in effect, makes it possible to spin up a virtual war room that enables the members of an IT organization to collaborate until the issue of the moment is resolved.

    The speed at which any IT issue is resolved has become a crucial metric. Given the complexity of IT environments, it’s inevitable that incidents will arise. Ideally, IT teams would discover those issues before they disrupt the business. However, in reality, the metrics organizations use to evaluate the effectiveness of a DevOps team are, unfairly or not, how quickly IT issues are resolved once they are discovered. The longer it takes for a distributed DevOps team to marshal its resources the more likely it becomes that team will be perceived as being inefficient—regardless of what level of application availability is actually achieved.

    Savvy DevOps teams, of course, have implemented a wide range of incident management best practices that create a level of muscle memory for responding to a wide range of issues. The trouble is there will always be unforeseen issues that arise. DevOps teams, as a result, need to find ways to quickly focus on an issue that, for one reason or another, were simply too unpredictable to anticipate; they need to expect the unexpected.