devopsy.kinsta.cloud

Tag: gitops

  • The Argo Project: Making GitOps Practical

    The Argo Project: Making GitOps Practical

    If you’re a DevOps engineer, chances are you have at least heard the name “Argo,” seen the friendly squid logo, and wondered: What the heck is this tool everyone is talking about? 

    In a nutshell, Argo is a popular open source tool that is making GitOps practical for anyone who uses Kubernetes. But it’s much more than that because, by enabling GitOps, Argo will also make Kubernetes environments more robust, secure and reliable than ever before. 

    Let’s find out why the combination of Argo, GitOps and good DevOps engineers is a recipe for production panacea.

    What is GitOps and Will it Replace DevOps?

    GitOps is a new(er) process and paradigm used by development teams to manage infrastructure and deploy applications. The “Git” in GitOps refers to the popular open source version control system. GitOps uses Git as a single source of information for declarative configuration. Based on this declarative configuration, it works its magic.

    GitOps automatically manages the configuration and deployment of infrastructure using Git pull requests. When development teams make a change to Git configuration, a GitOps agent deployed in the environment automatically reconciles the changes with the live state. Every change to the live environment is captured in the Git repository, so teams have unprecedented visibility and auditability over system changes. Most importantly, it is trivial to roll back to a previous working version in case of fires in production.

    GitOps builds on the developer experience, enabling teams to manage their infrastructure using the same tools and processes they use to develop software, extending those tools to the realm of software deployment and infrastructure management.

    While some have touted GitOps as “DevOps 2.0”, many experts disagree, saying that GitOps is not a better version of DevOps, nor is it something that will replace DevOps altogether. GitOps is a technical practice that can be highly beneficial to DevOps teams. The DevOps engineer will not, as some fear, be replaced by a Git-triggered robot. But GitOps will make her and the developers she supports happier and more productive.

    What is the Argo Project?

    Argo is an open source project hosted by the Cloud Native Computing Foundation (CNCF) for building and managing continuous delivery workflows in Kubernetes, GitOps style. The unique aspect of Argo is that it is Kubernetes-native—designed from the ground up for a modern containerized environment.

    The Argo project consists of four main sub-projects:

    • Argo CD—A GitOps continuous delivery engine with a powerful GUI and CLI.
    • Argo Workflows—A workflow engine that supports workflows and directed acyclic graphs (DAGs) in which each process stage is a container.
    • Argo Rollouts—An advanced Kubernetes deployment engine that supports progressive delivery strategies like canary and blue/green deployments, which are difficult to implement in plain Kubernetes.
    • Argo Events—A Kubernetes event-based dependency management system, which can be used to trigger automated workflows in CI/CD pipelines.

    Requirements for Kubernetes in Production

    Kubernetes is an orchestration platform that can help deploy, scale, and manage containerized applications. It can manage containers and the underlying infrastructure—scheduling containers and managing their workloads. This capability integrates software development and platform management to improve application management and portability. 

    Kubernetes in production can offer many advantages. For example, it makes it possible to release frequent changes to a high-availability application while keeping them online and enables you to move applications from one cluster to another. However, you need to configure Kubernetes correctly to realize its potential.

    Configuring Kubernetes for production involves implementing requirements according to each use case. As a result, what constitutes “production ready” in one Kubernetes project may vary greatly between different use cases. However, there are still minimum requirements that should be applied to most use cases. Here are key requirements for Kubernetes in production:

    • Security—When securing Kubernetes for production, you need to minimize the attack surface. This means locking down the Kubernetes control plane, applications, and their associated data. You need to keep secrets secured, harden nodes, use role-based access control (RBAC), and keep your Kubernetes version up-to-date with the latest security updates.
    • Portability and scalability—You can maximize cluster performance by maintaining portability and scalability. There are many Kubernetes capabilities that can help you achieve this goal, including self-healing nodes and auto scaling the infrastructure.
    • Deployment velocity—Kubernetes makes infrastructure and applications management faster by automating related processes. You can use Kubernetes’ automated features with GitOps to increase deployment velocity across your CI/CD pipelines.
    • Disaster recovery—You can significantly minimize your mean time to recovery (MTTR) by using Kubernetes and GitOps together. GitOps can help you reduce MTTR to minutes and quickly and easily recreate your container infrastructure in case of a complete cluster meltdown. 
    • Persistence—To support stateful production applications you will need to configure Kubernetes persistent volumes, provision storage devices with multiple storage tiers, and ensure they are efficiently utilized.
    • Observability—Observability enables you to gain a high-level view into running services, helping you make informed decisions before deployment to mitigate risk. GitOps can help you continuously monitor Kubernetes deployments, ensuring you can act immediately when issues occur.

    Benefits of Argo Tools for Production Environments

    Argo CD

    Argo CD is a continuous delivery (CD) tool native to Kubernetes. While most CD tools only support push-based deployment, Argo CD works in pull mode, retrieving updated code from a Git repository and deploying it directly to Kubernetes resources. This allows developers to manage infrastructure configuration and application updates in one system.

    Argo CD key features include:

    • Automatically synchronizing application state in a Kubernetes cluster with the current version of declarative configuration in a Git repository (==GitOps).
    • Ability to visualize deployment issues and detect and fix misconfigurations.
    • Convenient web-based GUI and CLI.
    • Role-Based Access Control (RBAC) and single sign-on (SSO), including login via GitHub, LinkedIn, Microsoft, etc.
    • Support for webhooks triggering actions in GitLab, GitHub, and BitBucket.

    How does Argo CD make Kubernetes production ready?

    GitOps can help you achieve a true GitOps workflows in Kubernetes, which provides the following benefits in production environments:

    • Minimize failed deployments and rapidly recover from downtime—Argo CD lets you instantly revert and rollback easily if anything goes wrong in production, reducing recovery time to seconds. 
    • Built-in auditing history—Argo CD ensures that every change to a Kubernetes cluster ia audited in the form of a Git configuration change log. This provides a complete audit trail of who changed what in the cluster, enabling consistent operations and supporting compliance requirements. 
    • Greater visibility on the lifecycle of developed features—Argo CD provides traceability from every change to infrastructure or applications directly to the code commit that caused those changes.
    • Improved security—Argo CD provides strong security guarantees, leveraging strong cryptography to manage changes and verify authorship, and creating a complete separation between CI environments and production systems, eliminating many types of supply chain attacks.

    Argo Rollouts

    Argo Rollouts is a set of Kubernetes controllers and CRDs that provides advanced deployment features such as blue/green and canary deployments, experiments, and progressive delivery in Kubernetes environments. It can integrate with input controllers and service meshes to incrementally transition traffic to new versions during updates. 

    A key capability of Argo Rollouts is that it can query and interpret metrics from many sources to validate that a deployment works properly and perform automatic upgrades or rollbacks.

    How do Argo Rollouts make Kubernetes production ready?

    By default, Kubernetes provides the Deployment object which updates applications using the “rolling update” strategy. In large production environments, rolling updates are considered too risky because there is no control over rollout speed and no way to automatically roll back in case of failure.

    Another limitation is that out of the box, Kubernetes deployments cannot query external metrics, such as real user measurements of performance or engagement, to determine the success of a deployment. 

    Argo Rollouts can address all these challenges and enable full progressive delivery in a Kubernetes environment, without requiring complex configuration.

    Argo Workflows

    Argo Workflows is an open source container-native workflow engine for orchestrating parallel tasks on Kubernetes. Argo workflows are implemented as Kubernetes Custom Resource Definitions (CRDs).

    Key features of Argo Workflows include:

    • Defining workflows where each step in the workflow is a container.
    • Modeling multi-step workflows as sets of actions or DAGs to capture dependencies.
    • Running CI/CD pipelines natively on Kubernetes without configuring complex software development products.

    How do Argo Workflows make Kubernetes production ready?

    Argo Workflows lets you automate production workflows in any Kubernetes environment, whether on-premises, hybrid cloud, or multi-cloud. It eliminates manual processes and ensures automation is represented as declarative configuration, making it consistent, repeatable, easy to rollback, and easy to troubleshoot.

    Argo Events

    Argo Events is a Kubernetes event-based dependency manager that defines multiple dependencies for different event sources (e.g. webhook, S3, schedule, stream, etc.) and Kubernetes objects after event dependencies are successfully resolved. Help the trigger.

    Argo events by themselves are not very useful. To deliver value, you need to integrate with a system that can execute workflow steps. So you can use Argo workflows to set Argo events. This helps orchestrate parallel Kubernetes jobs.

    How do Argo Events make Kubernetes production-ready?

    Argo Events provides several capabilities that enable dependency management in production Kubernetes environment:

    • Manage dependencies for various event sources.
    • Ability to customize business-level constraint logic to resolve event dependencies.
    • Manage everything from simple linear real-time dependencies to complex multi-source batch job dependencies.
    • Compliance with the CloudEvents standard.
    • Ability to manage event sources at runtime.

    Conclusion

    In this article, I covered the basics of GitOps and the Argo project and showed how each of the Argo project’s key modules can help make Kubernetes production-ready with minimal effort from developers and DevOps engineers. The magic of Argo is its simplicity and the way it fits seamlessly into existing Kubernetes workflows. I hope this will encourage you to try it out and see if it can help with your organization’s production issues.

  • It is Time to Secure Git

    It is Time to Secure Git

    At this point, we’ve got a ton of experience with the bits of Git that we use. And locking down Git is well-documented. Okay, it is documented; we can say that, at least. Interestingly, much of the ‘how to secure Git’ information out there is actually ‘how to keep critical information out of Git,’ and that is the problem. ‘Critical information’ always includes warnings to “not put configuration information into Git. Ever.” And that has been the default way to lock down Git for nearly its entire existence.

    Then along comes GitOps. What is its purpose, in a nutshell? To put that critical information into Git. So we have been waltzing along; only a fraction of us actually locking down our implementation because we were making sure the data in Git, while valuable, was not full of bits of information that would be useful in compromising systems. But now it will be. In fact, in GitOps, nearly everything is one of those bits of information useful to attackers.

    All of which means that it is time to get serious about Git security. Not content, but actual Git security. I won’t go crazy with links to lockdown documents here—it is a vastly different process to lock down a local Git repository versus a remote one and it is drastically different when you’re locking down a service like GitHub or GitLab (actually, some of the work is done for you with a SaaS vendor) … so, you’ll have to look it up yourself, based on your Git implementation. But there are documents out there that walk you through it; you’ll just have to make sure you find the ones that say, “How to lock down Git repositories” and not “How to keep privileged data out of Git repositories.”

    But whatever you do, stop putting it off. Now, if a neer-do-well gets into your repository, it won’t just be source code for various apps that are leaked—and that is certainly bad enough, even if your organization is not a software organization—it will be infrastructure information that can be used and abused. So make certain that information is protected in every possible way. Point security at Git and tell them, “We know you are insanely busy, but this is the most important thing to protect if we are going to implement GitOps.” Because it quite literally is. If database configuration, network configs and possibly even login info are to be stored in Git, locking it down (both local and remote copies) is imperative. You wouldn’t leave that info in plain text on a centralized data store and count on existing protections. So don’t do the equivalent with Git.

    If GitOps isn’t in your immediate future, that’s still your source code full of vulnerability advertisements out there, so at least ask your security team to give your Git install/policies/config a review. Even as a developer, I am less worked up about source code repositories than configuration repositories. There are several reasons for that, but the big one is you all. The bulk of organizations just don’t seem that worked up about source code repository security, but that changes when operations is asked about the list of holes in the firewall being moved into Git—and being configurable from Git (as just one example).

    So, keep kicking it! You all are making, delivering and securing a metric ton of software, and the complexity of our environments did not decrease after the remote work wave hit. Yet, you are still rocking it. May you have enough staff to do the job and applicants that come looking to work with you over competitors. And keep showing the world that software is manageable. But secure your Git.

  • Source Code Management Best Practices

    Source Code Management Best Practices

    Source code is, of course, the foundation of any software development practice. Managing that source code is the first task tackled by modern software development pipelines, with all subsequent stages of the pipeline dependent on the source code for their success and functionality. Thus, it is critical that source code is properly managed without introducing bottlenecks or defects into the delivery pipeline. To do so, there are some best practices to follow when interacting with source code.

    Regularly Commit Changes

    While this may seem obvious, commits often are done sparingly, as most developers tend to commit only after a significant change. The commit frequency depends on the workflow and it is acceptable to commit only after major modifications if that commit triggers another task.

    However, it is better to commit changes—even minor ones—regularly. It not only keeps an auditable history of changes but also enables easy comparisons between commits for troubleshooting and ensures that the code is up to date with the version control system (VCS). However, it does not mean that users should make commits for every small change; instead, adapt a development style that includes frequent commits as part of the workflow. Even if there are many commits, most VSCs provides users with the option to combine multiple commits to a single commit using a rebase for better management.

    Always Ensure That the Repository is Up to Date

    Most of the time, development is a team effort with multiple developers working on a single codebase. Thus, there will be frequent updates to the source code. So it is always advisable to use git pull or fetch to obtain an updated copy of the code before making any changes. This practice not only mitigates merge conflicts but also helps users to avoid errors while pushing the code, such as ‘failed to push some refs to‘ errors which occur when the local copy of the repository is behind the remote.

    Create Detailed Commit Messages

    Commit messages should be meaningful and include the ability for anyone reading the message to grasp the reason behind the commit quickly. It does not mean that users should create lengthy commit messages; instead, they should describe the commit in a concise yet meaningful manner. The commit message should include why the code was committed and not how it can be identified by comparing the files.

    A good commit message can include details such as bug fix ID or requirement ID, which reference the specifications of that code change in project management platforms like Jira and Azure Boards.

    Review Changes Before Committing

    Always make sure to review the code before making commits—this is imperative if you are committing to a shared repository. Typically, code reviews are a part of the delivery pipeline before merging code with the master branch or a specific repository via a review system or a pull request.

    Code reviews put a second set of eyes on code modifications and help detect any issues in the code. It also acts as a quality control mechanism that helps organically improve the overall code quality while helping to increase the awareness of each code change.

    Implement a Proper Workflow

    A proper workflow with standardized practices must be implemented across the development team even while using a source code management (SCM) tool. Some considerations for a standardized workflow:

    • Decide on a branching strategy and branching best practices like naming conventions for specific development types. For example, using a prefix to indicate the type of development feature or bugfix
    • Ensure proper use of scopes to decide who has access to each repository
    • Make sure the review workflow includes which users have the final say in the review process and proper authorization to merge the changes.

    A properly defined workflow describes the complete source code management cycle from creating new branches to making changes and, finally, merging the code to the master branch after review. This process helps organizations simplify the overall code management by pushing users to follow a specific workflow. This can also help easily onboard new developers.

    Implement Proper Security Practices

    This practice goes hand-in-hand with proper workflows, as security will be one of the primary considerations when defining the workflow. It should start from developer authentication and authorization, which controls who has access and what areas of the source code they have access to. Additionally, security practices should provide visibility into all the user activities with auditable trails to pinpoint any unusual activity in the source code.

    Security practices coupled with a good backup and disaster recovery strategy are vital for the continuous operation of the development pipeline.

    These best practices can help developers manage their source code securely and efficiently. A properly implemented source code management system will lead to fewer issues in the preceding stages of the pipeline and ensure smoother operation in the long run.

  • Best of 2021 – The Next Evolutions of DevOps

    Best of 2021 – The Next Evolutions of DevOps

    As we close out 2021, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the seventh in our series of the Best of 2021.

    As society continues to recover from the global COVID-19 pandemic and some semblance of normal starts to return, technology continues to move forward. The pandemic certainly changed what normal means and people, processes and technologies all have adapted and evolved to meet those challenges. The narrative that the pandemic caused of your organization’s digital transformation is true for many practitioners. With 2022 just around the corner, DevOps is on the cusp of maturation of previously bleeding-edge paradigms and a focus on engineering efficiency. Here are five DevOps evolutions you should keep an eye on going forward.

    Engineering Efficiency is Front and Center 

    Engineering efficiency is a particularly broad umbrella term. At the foundational level, engineering efficiency focuses on making someone (for example, an engineer) more productive. There are a multitude of disciplines that intersect, from organizational design to engineering/developer experience. Engineering efficiency is increasingly critical for organizations today. The pandemic had two major impacts on the technology sector in terms of resources and staffing. The first, at the start of the pandemic, was the unprecedented and unpredictable availability of resources; physical locations were being shut down because of the medical severity of the pandemic and the “getting hit by a bus” factor was playing out in real life. Second, during the recovery period, the Great Resignation began, and with it, a fight for resources that has become increasingly intense. Both have meant that the availability of resources, especially ramped resources, is scarce. 

    The Spotify model of tribes/guilds of engineering resources is a promising evolution; folks can frequently move around, reducing the toil that engineering resources face and allowing for engineering resources to not only ramp up more quickly but also increasing engagement and retention. The industry continues to march toward standardization in areas that were typically bespoke, such as the release process, and I think we’ll see that continue to evolve, as well.

    Git is Ubiquitous

    Leveraging a source code management (SCM) solution used to be reserved for software engineers. But as the proliferation of “something-as-code” has touched multiple levels of the technology stack from networking, storage and, ultimately, a development pipeline, the march of operations-focused engineers adopting many of the traits of software engineers continues. Having iteration in your infrastructure stack with more ephemeral/disposable infrastructure is becoming the norm. Saving and packaging the multitude of configurations for the “something-as-code” source control is a natural area of evolution. 

    Building off of source control are the package managers—your Docker registries, Helm chart repositories, etc.—for more deployable artifacts; that said, getting those artifacts to production is a process. Because Git is becoming ubiquitous across the technology landscape, this leads to the continued adoption of GitOps. Since the seminal piece was written by Weaveworks in 2017 defining the GitOps paradigm, leveraging GitOps is seen as a viable paradigm for many organizations five years later, especially when starting with greenfield initiatives. 

    Kubernetes is No Longer Bleeding Edge

    In 2022, Kubernetes will celebrate the eighth birthday of its first commits on GitHub. Taking a jog down technology memory lane, eight years before Kubernetes (in 2006), VMware was still a private company and it would be a few years before vSphere was even released (remember how it felt to run a workload on a virtual machine in 2014?). The Kubernetes ecosystem still moves quickly, but placing workloads on Kubernetes is not a novel concept anymore. As application infrastructure and architecture have adopted the Kubernetes way (in other words, being idempotent and ephemeral), there is maturity around running a suitable workload on Kubernetes.  

    Kubernetes by design is highly pluggable. If you do not like the opinion or implementation of something inside Kubernetes, you can replace that opinion. Don’t like how Kubernetes handles ingress traffic? Then choose from one of the many ingress controllers available today. That is just one example of dozens of pluggable areas. Because of this, Kubernetes is viewed as a dynamic, non-static resource. Using Kubernetes takes trial and error and getting your cluster to be robust, performant and reliable is an ongoing journey that requires ongoing iteration. 

    Authors Can Be the Enforcers

    There is no question that in a modern software stack, we are inundated with data. The metrics soup and the continued science of and exploration of observability (metrics, traces, logs) provide us with a lot to work with. Making decisions, even automated ones, based on that data continues to be critical. With the rise of site reliability engineering (SRE) practices, reacting or anticipating spikes in usage, failures and even security-related items are par for the course for modern teams. 

    These decisions can be authored into policies that specific platforms understand, such as autoscaling rules for a cloud vendor or something like Open Policy Agent in the Kubernetes ecosystem. With so many items shifting left toward the development team, finding the right resource, skillset or institutional knowledge to author these rules can involve the input of several teams. Because of the rise of “something-as-code,” regardless of whether the author of these rules is a software engineer, DevOps engineer, platform engineer or anyone in between, that author has the ability to enforce. 

    Shifting Left … But Complexity Shifts Left, Too

    The old adage is that complexity is like an abacus: You can shift complexity around, but it never really goes away. With the movement to shift responsibility left to development teams, this also means that associated complexity is shifting to the development teams. Modern platform engineering teams provide the infrastructure (compliant Kubernetes clusters) to teams and any workload that is run on those clusters is up to the development team that owns it.  Typically, development teams then focus on features and functionality. Managing lots of non-functional requirements—and even core infrastructure requirements such as networking—can be a burden; think about how your organization would handle a service mesh.

    If you are a DevOps or platform engineer, making your internal customers—your development teams—successful is a great goal to work toward. Crucial to this is disseminating expertise. This can be in the form of automation and education. A common practice with the DevSecOps movement is to have some sort of scanning step as part of the build or deployment process, disseminating the internals as far as how the scan is performed, what happens if something is found, etc. Gaining internal adoption is a journey and having a good developer experience built around clarity and stability are important. 

    Evolution in 2022

    Even with all of the challenges, bumps and learnings that the last year has given us, technology continues to evolve and advance and even lower the barriers of entry to continually become more inclusive. Focusing on improving engineering efficiency and reducing toil will allow for more participation and lower the barriers to entry.

     

  • Extending GitOps to the Enterprise

    Extending GitOps to the Enterprise

    GitOps is a relatively new term, but it continues to gain momentum as more organizations embrace the paradigm. In DevOps, GitOps lands on the software engineering side of the development and operations continuum. Even though GitOps is more likely to be adopted by smaller organizations, larger enterprises can successfully implement GitOps with just a few tweaks to the overarching framework.

    Let’s start at the beginning. Wouldn’t it be great if we could combine DevOps and GitOps approaches and use the Git distributed version control system as the ultimate source of truth? Then, when there’s a dispute over the correct state, people know where to go for the correct version.


     

    In GitOps, the system’s desired configuration is stored in a revision control system such as Git. Any difference between the desired state stored in Git and the system’s actual state indicates to DevOps teams that not all changes have been deployed. These changes can be reviewed and approved through standard revision control processes such as pull requests (PRs). When a PR is approved and merged to the main branch, an operator software process is responsible for changing the system’s current state to the desired state based on the configuration in Git.

    GitOps doesn’t require a particular set of tools, but the tools must:

    1. Operate according to the desired system state stored in Git.
    2. Detect differences between the desired and actual states.
    3. Perform required operations on the infrastructure to synchronize the actual and desired states. 

    In an ideal implementation of GitOps, manual changes to the system are not permitted and all changes to configuration must be made to files stored in Git. The infrastructure and operations engineers’ roles in a GitOps model shift from performing infrastructure changes and application deployments to developing and maintaining GitOps automation and helping teams review and approve changes through Git. 

    GitOps is great for non-critical environments, but enterprises are less likely to adopt GitOps due to a number of challenges facing GitOps users.

    Challenges with GitOps

    GitOps is applicable only to a subset of the software development life cycle (SDLC). This is important because GitOps tools are sometimes marketed as a one-size-fits-all solution that will solve all release problems—this is simply not true. First, GitOps requires that your deployment artifacts are already there. This means that tasks such as

    • Compiling code
    • Running unit/integration tests
    • Security scanning
    • Static analysis

    are not a concern of GitOps tools and are assumed to already be in place. GitOps also doesn’t address promotion of releases between environments.

    Auditability

    A default GitOps process is not fully auditable because of force push. Force push essentially allows users to remove any unwanted blocks of commits from Git history on the central repository.

    Business Approvals

    PRs are a typical approval model used in GitOps. Developers make changes, create PRs and then an approver may accept that PR. This would be recorded and a change would be deployed. A PR-based approval model assumes peers are already using Git and are familiar with PRs. 

    Generally this works great until we need to include business approvals. Business personnel may not be familiar with Git or PRs. If a team needs business approvals, they have to build a process on top of GitOps. This drastically increases lead time. 

    Governance 

    GitOps is dependent on a pure PR-approval based model, one in which the PR approver is responsible for reviews. After a PR is approved, there is no way to enforce enterprise policy rules. Enterprises need to control what end users can do on specific clusters to ensure those clusters are in compliance with policies to meet governance, legal requirements or enforce best practices.

    An Enterprise Approach to GitOps

    However, there are ways to solve this problem at the enterprise level.

    Alerting on Drift Detection 

    Enterprise customers get alerted whenever there is a drift in their repository and, based on that, customers can choose to approve or reject the drift. If rejected, the merged PR should be reverted to the previous state to ensure that Git is in sync with the desired state of the cluster.

    Use a Pipeline for Promotion of Releases Between Environments

    To promote an artifact between environments, users would have the ability to use a pipeline with deployment stages, performance monitoring and log analysis/approvals. 


     

    Let’s say the developer commits the source code and the GitOps deployment happens in the QA environment. The PR can be automatically approved upon meeting the policy standards set by the company. Then it can be pushed to a production environment.

    Governance 

    There are many reasons why developers and platform engineers integrate policy-as-code into their current GitOps processes. For one, this strategy helps to accelerate application development and deployment because it helps solve many of the change management hurdles that slow development pipelines.

    On the development side, policy-as-code helps app developers understand and abide by the company’s configuration, security and compliance policies. For example, a developer may not remember—or may have no reason to know—when deploying a load balancer onto Kubernetes in AWS is or is not sanctioned. Policy-as-code solves this problem automatically.

    The ability to run policies during GitOps deployments and adhere to those policies as a part of the deployment and upon any violations—alert, audit, notify and automatically roll back the changes from Git to the previous successful compliant state so that Git remains in sync with the cluster.

    GitOps + AIOps

    There is a notion that once GitOps is in place, everything will work seamlessly with no problems. Not so much; it is important to continually monitor deployment state to ensure everything stays as it should. That is where AIOps comes into picture.

    GitOps enables DevOps and development teams to see any manifest change, infrastructure change or artifact change. Post-deployment, the GitOps operator can instantly understand the changes in the service health, see what caused those changes and automate the process of investigating them using AIOps. If any severe anomalies are found, the operator can determine the current state of the system and, should anything differ, revert to the previous compliant Git commit. 

  • Iterative Adds Experiment Versioning to MLOps Platform

    Iterative Adds Experiment Versioning to MLOps Platform

    Iterative today added an experiment versioning capability to an open source platform for managing machine learning operations (MLOps) using GitOps workflows.

    Dmitry Petrov, Iterative CEO, said the latest version of the Data Versioning Control (DVC) platform makes it simpler to save, compare and reproduce machine learning (ML) experiments at scale without requiring organizations to set up an additional repository to track them. Instead, organizations can store those experiments alongside other software artifacts in a Git repository, he said.

    As a result, IT teams will no longer need to try to keep track of experiments using spreadsheets or notebook tools such as Jupyter, he noted. Organizations that are building AI models for applications constructed using DevOps workflows will also find it easier to comply with a wide range of existing and forthcoming compliance mandates, added Petrov.

    The DVC platform makes use of a Git-like interface to enable organizations to track multiple versions of data sets, models and pipelines across an MLOps workflow. That capability is being extended to make it easier to manage the experiments data science teams create during the construction of an AI model.

    Iterative is at the forefront of an emerging debate over how best to infuse AI models into applications. Providers of platforms used by data science teams are making a case for separate repositories for tracking the artifacts used to construct an AI model. Iterative, on the other hand, is arguing for the use of an AI model build framework that uses the same Git repositories where any other type of software artifact is stored and shared. Iterative’s argument is that, in effect, AI models are just another type of software artifact, noted Petrov.

    In addition to reducing the total cost of building AI models by reducing the number of platforms required, a Git-based approach also makes it simpler for DevOps and data science teams to collaborate because the DevOps team will have more visibility into what AI models will eventually need to be incorporated into an application.

    It may be a while before the development of AI models and applications fully converges. Data science teams today typically have defined their own workflow processes using a wide range of graphical tools. However, as it becomes obvious that almost every application is going to be infused with machine learning and deep learning algorithms to some degree, the need to bridge the current divide between DevOps and data science teams will become more acute.

    In the meantime, DevOps teams should assume that many more AI models are not only on the way but that those models will need to be continuously updated. Each AI model is constructed based on a set of assumptions; however, as more data becomes available, AI models are subject to drift that results in less accuracy over time. Organizations may even determine that an entire AI model needs to be replaced because the business conditions on which assumptions were made are no longer valid. One way or another, the updating and tuning of AI models is likely to soon become just another continuous process being managed via a DevOps workflow.

  • ArgoCon ’21: Get More Done with Kubernetes

    ArgoCon ’21: Get More Done with Kubernetes

    Argo is an open source suite of projects that helps DevOps teams deliver software more rapidly and safely. Argo Workflows, Argo CD, Argo Rollouts and Argo Events are open source tools for Kubernetes to run workflows, manage clusters and do GitOps right. On Dec. 8, the Argo community will come together at ArgoCon ’21 Virtual Event to share their Argo journey, some less known details about the project and offer a glimpse into what the future may hold.

    The agenda for ArgoCon ’21 features technical sessions, demos, deep-dives and case studies covering a variety of topics, including:

    • How GitOps workflows relate to DevOps movement and how it paves the way toward a successful DevOps adoption
    • How to use Argo Workflows, Events, CD and Rollouts to create a flexible, customizable developer experience for CI/CD
    • Argo CD production best practices and how to customize your Argo CD setup
    • Building medical grade AI with Argo Workflows
    • Bridging into Python ecosystem with cloud-native distributed ML pipelines
    • How to enable and empower teams to manage their own Argo-based CI/CD pipelines across multiple multi-tenant clusters
    • How scalable is Argo
    • GitOps-ifying your K8s management with Argo CD and Codefresh

    Speakers

    ArgoCon ’21 features outstanding DevOps and GitOps leaders from the Argo community, including:

    • Alex Collins, Argo project lead at Intuit
    • Alexander Matyushentsev, Argo project lead at Intuit
    • Anaïs Urlichs, SRE at Civo
    • Aya Ivtsan, senior computer scientist at Adobe
    • Christian Hernandez, senior principal technical marketing manager at Red Hat
    • Dan Garfield, co-founder and chief open source officer at Codefresh
    • Edgar Magana, senior principal software engineer at Splunk
    • Hannah Seligson, developer advocate at Codefresh
    • Hong Wang, founder and CEO at Akuity
    • Hui Kang, platform engineer at Salesforce
    • Ian McGraw, lead platform engineer at Arthur.ai
    • Jesse Suen, Argo project lead, co-founder and CTO at Akuity
    • J.P. Zivalich, CTO at Pipekit
    • Kelsey Hightower, principal engineer of Google Cloud Platform at Google Cloud
    • Mike Bowen, director open source program office, cloud native engineer at BlackRock
    • Omri Fima, chief architect at Diagnostic Robotics
    • Pratik Wadher, SVP, product development at Intuit
    • Ravi Lachhman, field CTO at Shipa
    • Ryan Umstead, BlackRock
    • Saad Malik, CTO and co-founder at Spectro Cloud
    • Sergio Méndez, DevOps engineer at Yalo
    • Severin Ryberg, senior infrastructure architect at ACCURE Battery Intelligence
    • Siamak Sadeghianfar, product manager at Red Hat
    • Venkata Gunapati, staff software engineer at Intuit
    • Yuan Tang, founding engineer at Akuity

    Giveaways

    There will be six sponsor giveaways at ArgoCon ’21:

    • A mouse pad and wireless charger giveaway by Red Hat
    • A $100 Amazon gift card raffle by Akuity
    • The Kubernetes 2021: Operationalizing for Scale study by Dimensional Research (sponsored by Spectro Cloud) is freely available for download
    • A $200 gift card raffle by Shipa
    • A brand new Quest 2 and Pipekit swag package giveaway
    • 15 Argo Codefresh branded Bluetooth speakers giveaway

    For more information on how to participate on the raffles and giveaways, visit the ArgoCon ’21 website.

    There will also be a virtual exhibit hall to interact with partner sponsors and download information. All attendees will receive a link to access the virtual environment on-demand once the live event is over. To see the full agenda and to register, please visit the ArgoCon ’21 website.

  • Top 10 DevOps Things to Be Thankful for in 2021

    Top 10 DevOps Things to Be Thankful for in 2021

    Thanksgiving has been observed in the U.S. since 1861 and was officially designated as an official holiday in 1863 by president Lincoln. Of course, the tradition itself traces its lineage to the early days of the Pilgrims.

    This past year, of course, has been one of recovery from a pandemic that caused the deaths of millions of people around the world. Vaccines have saved the lives of countless more, so there is much to be thankful for despite all our recent trials and tribulations. In that spirit, there are more than a few things for DevOps professionals, in particular, to be thankful for this year starting with:

    Digital business transformation: Organizations of all sizes have had to adjust their business models because of both major changes in buyer behavior and an ongoing supply chain crisis. DevOps teams are at the core of any major digital business transformation initiative, so the demand for DevOps expertise continues to outstrip the available supply. Going forward, it’s been predicted we’ll see more digital transformation in the next 10 years than in all the last 40 years combined. The vast majority of those initiatives would never see the light of day if it were not for the expertise of developers and engineers working hand-in-hand to build and deploy applications faster than ever.

    Low-code platforms: As the demand for applications to drive digital workflows spiked in the wake of the pandemic, professional developers relied more on low-code platforms to decrease the time required to build an application. While many complex applications will still be built using procedural code, it’s clear not nearly as many applications could have been deployed without relying on low-code platforms to build them. It remains to be seen how many of the next generations of low-code applications will be built by so-called citizen developers that may soon be flooding DevOps pipelines with application code.

    Microservices: As an architecture for building applications the core concept of employing loosely coupled services together to construct an application goes all the way back to when service-oriented applications (SOA) were expected to be the next big thing in the 1990s. Microservices have, of course, been around for several years themselves. However, with the rise of containers as a software artifact for building microservices, it’s only now becoming apparent that building an application constructed using microservices not only takes less time but is easier to maintain and also more resilient.

    Observability: As a concept, observability traces its lineage to linear dynamic systems. Observability in its most basic form measures how well the internal states of a system can be inferred based on knowledge of its external outputs. In the past year, a wide range of IT vendors introduced various types of observability platforms. These make it easier for DevOps teams to query machine data in a way that enables them to proactively discover the root cause of issues before they cause further disruption. That’s a major advance over monitoring platforms that typically only provide predefined metrics to identify when a specific platform or application is performing within expectations.

    Biden’s executive order: In the wake of a pernicious attack against IT service management software from SolarWinds, the Biden administration published a sweeping executive order that requires federal agencies to implement a wide range of security best practices. Foremost among them is guidance concerning the security of software supply chains; which resulted in many organizations reviewing their entire software supply chain to make sure similar breaches will not occur.

    DevSecOps: As an extension of any DevOps best practice, DevSecOps generally describes an overall shift left in terms of making developers more responsible for application security. While still in its relative infancy, DevSecOps is starting to gain traction as DevOps and cybersecurity teams look to bridge a cultural divide that is as wide as any in IT. The ultimate goal is to deliver more secure applications without reducing the speed at which they are being built and deployed.

    GitOps: At its core, GitOps is an operational framework that takes DevOps best practices used to automate application development and extends them to infrastructure. The basic idea is that the configuration software created to manage infrastructure-as-code is just another software artifact to be stored in a Git repository. The approach provides a means for IT teams to achieve a continuous delivery goal that, more than a decade after DevOps best practices were defined, remains elusive. It’s still early days as far as GitOps best practices are concerned, but this has been the year when GitOps become part of the mainstream DevOps conversation.

    Kubernetes: Simultaneously one the most powerful and most complex IT platforms ever to come down the proverbial enterprise IT pike, Kubernetes has emerged as a de facto standard for building and deploying container applications that can more efficiently scale up and down as required to consume IT infrastructure. The challenge (and the opportunity) now is to make Kubernetes much more accessible to mere mortal IT administrators that typically don’t have the programming skills of a site reliability engineer (SRE) and which are both hard to find and retain.

    GraphQL: Employed as a query language for application programming interfaces (APIs), GraphQL provides a more efficient means of accessing data than the REST APIs that have been employed by developers in the past. Originally developed by Facebook, GraphQL adoption is still in its infancy but it’s clear APIs based on GrapQL will one day be as plentiful as REST APIs. That’s especially critical for microservices-based applications that rely on APIs to access data across a highly distributed computing environment that requires a more efficient means for transferring data.

    Venture and Wall Street capital investments: Investment firms are helping fuel DevOps advances as they continue to make capital available for companies vying for their share of a market that is expected to see compound annual growth rates of 24.7% through 2026. There will, of course, eventually be a wave of consolidation as merger and acquisition activity inevitably ensues. In the meantime, however, there is a wide range of DevOps technologies that might not otherwise be as broadly adopted if it were not for all the cash being poured into DevOps companies.

    The most important thing, of course, is that so many DevOps professionals are still here to build and deploy applications. It’s been a trying year for all concerned; however, as the pandemic crisis continues to recede, we can all be thankful for the opportunities that lie ahead.

  • Highlights from GitOps Evolution Virtual Summit

    Highlights from GitOps Evolution Virtual Summit

    GitOps has become increasingly popular in the DevOps space. DevOps teams have been adopting GitOps in order to better deploy, manage and monitor Kubernetes clusters and applications. On Sept. 23, the DevOps community came together at GitOps Evolution Virtual Summit to explore ways in which organizations can use GitOps to solve real problems.

    The virtual event, hosted by MediaOps, offered expert-led sessions and panel discussions focused on a variety of topics, including:

    • GitOps and the evolution of DevOps
    • A developer’s guide to adopting GitOps
    • Metrics to measure in GitOps environments

    GitOps Evolution Virtual Summit is now available on-demand for up to 30 days after the live event. Register now to get exclusive access to the on-demand virtual environment.

    Meet the GitOps Evolution Leaders

    GitOps Evolution Virtual Summit featured an exceptional lineup of industry thought leaders and doers in the GitOps space, including:

    • Tracy Ragan, DeployHub CEO and CDF board member
    • Viktor Farcic, developer advocate at Upbound, member of the Google Developer Experts and a published author
    • Paul Curtis, principal solution architect at Weaveworks
    • Ravi Lachhman, evangelist at Harness
    • Jayne Groll, DevOps Institute CEO
    • Alan Shimel, MediaOps CEO

    GitOps Evolution Sessions

    GitOps – An Introduction, by Alan Shimel and Paul Curtis

    In this session, Paul and Alan discussed the four principles of GitOps, how an enhanced developer experience increases productivity, how to boost stability and reliability in Kubernetes environments and how to bake in security from the start.

    Automating the Human Side of GitOps, by Tracy Ragan

    GitOps requires many manual steps, from updating .yaml files for each new container image, and managing Pull Requests, Merges and Commits. In this session, Tracy explored how to move GitOps to a fully automated DevOps process and reviewed the basic architecture of GitOps, how it is managed manually and how GitOps needs to evolve to be part of the continuous delivery pipeline.

    How To Apply GitOps On Infrastructure, by Viktor Farcic

    During this talk, Viktor Farcic explored Crossplane combined with Flux as a potential solution for all our infrastructure needs and whether it fits into the broader Kubernetes ecosystem.

    GitOps Blind Spots – Introduction to Pragmatic GitOps, by Ravi Lachhman

    Check out Ravi’s session to learn how GitOps can be a continuum in a journey and introducing Pragmatic GitOps practices and approaches no matter where you are in your software development approaches. Ravi talked about common blind spots with embracing on a pure GitOps approach might leave to be solved for.

    The Evolution of GitOps Maturity for Business Success, by Paul Curtis

    In this session, Paul went on a four-step GitOps journey that started with operating a single cluster and application, to managing large deployments and fleets. He discussed the requirements and business benefits of each step and tied them back to improving Software Delivery and Operational (SDO) performance—including deployment frequency, shorter lead time, mean time to recovery and change failure rate.

    Fireside Chat: The People, Process, and Technology of GitOps, by Jayne Groll, Tracy Ragan and Viktor Farcic

    Join Jayne Groll, Tracy Ragan and Viktor Farcic for a panel discussion probing further into the people, process and technology of GitOps – including what’s next, challenges to culture and theory versus practical implementation.

    For more information and to access the on-demand environment, visit the GitOps Evolution Virtual Summit website.

  • How to Apply GitOps on Infrastructure

    How to Apply GitOps on Infrastructure

    GitOps has become more than a buzzword in the DevOps community. It consists of a set of best practices designed to better deploy, manage and monitor Kubernetes clusters and applications. By adopting GitOps everything becomes code, therefore empowering DevOps teams to control every aspect of the development process from beginning to end.

    Viktor Farcic, developer advocate at Upbound, member of the Google Developer Experts and a published author, has been doing GitOps even before the term was coined by Alexis Richardson in 2017. Viktor believes that even though many developers have been doing GitOps for a while, defining what it really is and giving GitOps a name is what has helped with its rapid adoption. 

    Viktor is one of our outstanding speakers presenting at GitOps Evolution Virtual Summit. During his session, Viktor will talk about GitOps and explore Crossplane combined with Flux as a potential solution for all infrastructure needs and to see whether it fits into the broader Kubernetes ecosystem.

    Crossplane is an open-source project that enables engineers to use Kubernetes to provision and manage infrastructure, services and applications. Combined with Flux, it allows us to apply GitOps principles to constantly monitor infrastructure and fix any drifts there are between the current and desired states.

    Our very own Jackie Whetzel sat down with Viktor Farcic for a quick interview on GitOps and his upcoming session. Check out a clip of the interview below.

    To learn more about how to apply GitOps on infrastructure join Viktor’s session at GitOps Evolution Virtual Summit on Sept. 23. To check out the full agenda or to register for free, visit the GitOps Evolution Virtual Summit website.