devopsy.kinsta.cloud

Tag: gitops

  • 5 Effective Git Workflows to Streamline Your Development Process

    5 Effective Git Workflows to Streamline Your Development Process

    In the fast-paced world of software development, efficient collaboration and version control are vital for successful project management. Git, a distributed version control system, offers various workflows to facilitate seamless collaboration among team members. Each workflow is designed to address specific development needs and enhance productivity. In this blog post, we will explore five popular Git workflows and discuss their significance in the software development process. 

    Centralized Workflow

    The centralized workflow is a simple and straightforward approach commonly used in small teams or projects. In this workflow, a central repository acts as a single source of truth for the project. Developers clone the repository, make changes in their local branches and push their changes to the central repository for integration. This workflow is easy to understand and implement, making it ideal for beginners. It promotes a linear development process, making it easier to track changes and maintain code consistency. 

    Feature Branch Workflow

    The feature branch workflow is widely adopted in larger development teams. It enables parallel development and promotes the isolation of work. Each feature or task in this workflow is given a specific branch. Developers work on their features, branch off from the main branch, and submit updates to the remote repository. Once the feature is complete, it undergoes code review and is merged back into the main branch. This workflow allows for better collaboration as developers can work independently on separate branches without interfering with each other’s progress. 

    Gitflow Workflow

    The gitflow workflow is a robust and well-defined branching model that is particularly useful for projects with multiple release cycles and long-term maintenance. It incorporates two main branches: The “master” branch for stable releases and the “develop” branch for ongoing development. Developers create feature branches from the “develop” branch, work on their features, and merge them back into “develop” for integration testing. Once the code in “develop” is ready, it is merged into the “master” branch for a new release. This workflow ensures a structured and controlled release process, making it easier to manage versioning and hotfixes. 

    Forking Workflow

    The forking workflow is commonly used in open source projects and promotes collaboration among multiple contributors. In this workflow, each developer creates a personal fork of the main repository. They make changes in their forked repository, create a pull request to propose changes and undergo a review process. The changes can then be merged into the main repository by the project maintainers. This workflow allows for a clear separation of duties and provides a controlled mechanism for accepting contributions from external developers. 

    Release Flow Workflow

    The release flow workflow is well-suited for projects that require frequent releases and hotfixes. It combines the concepts of the feature branch and gitflow workflows. The main difference lies in the way releases are managed. In this workflow, a dedicated release branch is created for each release. Once the release is ready, it is merged into both the “master” and “develop” branches. This workflow allows for efficient bug fixes and maintenance, ensuring that critical updates are quickly propagated to the main development branch. 

    Why Are These Workflows Needed? 

    Efficient Collaboration: Git workflows provide structure and guidelines for collaboration, enabling developers to work in parallel without conflicts. They facilitate seamless communication and code integration, fostering teamwork and productivity. 

    Code Consistency and Quality: By following established workflows, developers can ensure code consistency and maintain high-quality standards. Workflows enable code review processes, catching potential issues early and reducing the risk of introducing bugs into the main codebase. 

    Traceability and Version Control: Git workflows provide a clear history of changes, allowing developers to trace back and understand the evolution of the codebase. They enable easy rollbacks and provide a reliable version control system, safeguarding against accidental data loss. 

    Streamlined Releases and Hotfixes: Workflows like gitflow and release flow are specifically designed to streamline the release process, making it easier to manage versioning, hotfixes, and long-term maintenance. These workflows ensure that stable releases are delivered promptly and critical updates are quickly deployed. 

    Conclusion 

    Git workflows are powerful tools that help streamline the software development process. Whether you’re a small team or a large enterprise, adopting the right workflow can significantly improve collaboration, code quality, and overall productivity. From the centralized simplicity to the structured gitflow model, each workflow serves a specific purpose and caters to different project requirements. By leveraging the appropriate workflow, developers can navigate the complexities of version control and empower their teams to deliver exceptional software. 

  • The DevOps Pendulum: Agility Vs. Control

    The DevOps Pendulum: Agility Vs. Control

    Managing changes to cloud assets is a universal pain felt by many engineering leaders today, despite all of the advancements in tooling and practices like GitOps. This is because, in reality, it is simply impossible to always have everything completely locked down—we don’t live in a utopia with zero incidents. If your engineering organization prevents any changes from being made through the cloud console or to your infrastructure-as-code (IaC) without complying with strict GitOps practices or change management processes via CI/CD, then it’s likely you have very frustrated developers—developers who can’t troubleshoot or debug in real-time and who have little control in a real-world incident.

    Engineering, like everything else in life, is all about balance.

    The Pendulum Swings

    The pandemic created a whole new mindset and practice for managing distributed and high velocity engineering and operations. Overnight, companies that were not built for remote ways of working had to continue their operations in a global, distributed and asynchronous way that they weren’t completely familiar with. This required a new way to think about software delivery—and it accelerated the DevOps practices that support that delivery. Self-service infrastructure removed barriers for developers to ensure continued performance and velocity.

    At the same time, your cloud cannot be the wild west with everyone creating bespoke infrastructure. It becomes impossible to manage, and misconfigurations can be risky. Guardrails and policy automation have become a hot topic. Today, with tech markets depressed and cloud costs escalating, there seems to be a growing trend toward locking things down again, even at the risk of frustrating developers.

    This begs the question: How can you achieve unencumbered infrastructure for your developers while simultaneously following policies and best practices for compliance, risk and cost? There is a way to find balance.

    Like many aspects of security, we’ve learned that when constraints and barriers are too high, users ultimately find ways to bypass them. This is true for operations, as well. While sometimes it might seem easier to lock everything down than design a better and more balanced way to enable developers to move fast, eventually this approach backfires. This is the exact same evolution the application and cloud-native security industry are undergoing right now.  All the guardrails and controls applied have created too much friction in development processes and developers ultimately bypass them.

    Real-Time Insight is Key

    CloudOps can learn a lot from the disruption the security industry is going through today. In the same way that point-in-time security has been rendered completely useless, non-real-time alerting or eventual alerting to infrastructure drift just won’t cut it when managing an ephemeral cloud. What is really needed is the same kind of real-time and continuous scanning of cloud assets and IaC, similar to what we apply to our systems through monitoring and observability. These solutions became an essential backbone of our business to ensure continuous operations and availability of cloud services.

    As we embrace IaC and the benefits it brings, everything-as-code unlocks greater agility and visibility, enabling you to automatically remediate without locking everything down. As DevOps says, “fail forward and fail fast.” Rather than focusing on never making a mistake, focus on how to fix it immediately. 

    The Solution

    By providing a continuous comparison of actual cloud assets with their desired state through IaC and GitOps, it’s possible to surface configuration drift and policy violations immediately, much like any other kind of breach or major system failure. Failures and incidents are inevitable. It’s unrealistic, and even dangerous, to build systems with an inherent underlying design that prevents you from changing something at the cloud console at 2:00 A.M. during an outage.

    With a locked-down approach, a developer would need to wait for the change management approval during a high-pressure production incident, oftentimes at 3:00 A.M. (because the incident gods of always make them happen in the middle of the night) or on a weekend. We don’t need to look much further than the catastrophic 40-minute and $440 million Knight Capital incident to understand that sometimes time is not on our side and that a delay can have severe consequences.

    Cloud drift has become equally troubling as uptime or any other mission-critical aspect of continuous business operations, and that’s why we need to apply the same principles for monitoring drift in real-time as we do to our CPU and load. This will make it possible to be alerted when your cloud and your IaC are no longer aligned and even flag issues that can be prevented in real-time. You can automate ticket creation and escalation and also triage whether this is something that needs to be fixed in real-time with remediation suggestions provided or open a ticket to fix it later. The choice should be yours, not made by an administrator who has a lot less context about the systems and the ultimate business impact.

    It is Not ‘One and Done’

    Some vendors preach codifying your cloud into IaC and then locking it down so that you’ll never have to do it again. This feels like deja vu; we’ve definitely heard this before. It sounds a lot like the ‘write once, deploy anywhere’ JVM promise that never quite panned out. 

    By going down this path, you’re resigning yourself to never adding new cloud assets or changing your cloud configurations. If there’s one thing we’ve learned about the cloud, it’s that it’s a constantly moving target.  (Read what cloud asset management can learn from the world of finance—another industry in constant flux). If this didn’t work decades ago with the JVM, it certainly won’t work in a dynamic and constantly changing environment like the cloud. That’s a fool’s journey.

    Continuous Cloud Up to Code

    There are major downsides of locking down agility and velocity. This is hard-earned advice, witnessed firsthand from tools like legacy CMDB and startups who have forgotten that cloud is ephemeral—and possibly the reason why we all use it in the first place! So if our cloud is constantly changing and is a continuously moving target, then shouldn’t our tools be built in just the same way to keep up? Solutions built for the rapidly evolving cloud-native era must continuously look for new, unmanaged and changed assets and ensure their cloud infrastructure always follows policies and regulatory standards.  

    Once we are able to find the balance between agility and control, we’ll be able to tap into the benefits of speed and safety (as DORA research shows time and again) that are the benchmark of high-performing teams. Automating cloud asset management can provide governance, policies and control while still enabling the velocity and agility engineering organizations today require. 

    Agility Versus Control

    There’s a minority of people who claim they don’t worry about cloud asset configuration drift because they have everything completely locked down. Zero changes can be done at the cloud console or to IaC without going through GitOps for strict change management processes via CI/CD. I bet they have happy developers—not!

    Life is All About Balance

    Throughout the pandemic, organizations had to make developers happy for fear they would leave. So DevOps went full tilt aiming for self-serve infrastructure for developers to remove barriers to speed and velocity. Now, with tech markets depressed, maybe it’s easier to lock things down even if that risks losing engineers.

    As we embrace IaC and its benefits, you need not live in the stone age of locking everything down. How can you achieve unencumbered infrastructure for your developers while following policies and best practices for compliance, risk and cost? There is a way. 

    The solution? Real-time continuous scanning of your cloud assets and your IaC with continuous comparison can surface configuration drift and policy violations immediately. Emergencies are inevitable. It’s bold/stupid/foolish to think you’ll never need to change something at the cloud console during an emergency or that a developer will wait for change management when production is down at 3:00 A.M. on a weekend. It happens, and you must deal with it. 

  • Extending the GitOps Pipeline: DevSecOps and Trusted Application Delivery

    Extending the GitOps Pipeline: DevSecOps and Trusted Application Delivery

    We live in a fast-paced world of rapid changes and uncertainty where every aspect of our daily lives changes often, including the way we work and communicate. Organizations must deliver value to their customers at a much faster pace than ever before to thrive in such an environment. However, the decentralized nature of cloud infrastructure expands the threat landscape and poses several security risks that enterprises should address in light of recent data breaches.

    Organizations must combat security threats and address challenges arising from cloud misconfigurations. Several recent tools, strategies, practices and technologies have emerged that can help you handle such security threats. This article discusses how the fusion of DevSecOps and trusted application delivery can extend the GitOps pipeline and add business value.

    What is GitOps?

    Coined by Weaveworks in 2017, GitOps thrives on a source code management system built on Git and adheres to the belief that Git is the single and definitive source of truth. Since all changes are stored in Git, anyone can see the audit trail if necessary. This enables you to use Git-based version control to manage your application’s code, environment and processes.

    Figure 1: CI/CD with Git-based Version Control

    GitOps leverages Git repositories as the one and only source of truth and embraces DevOps and infrastructure-as-code (IaC) best practices. Using GitOps, your developers can spend more time developing applications than configuring or maintaining Kubernetes clusters.

    GitOps strategies are gaining traction with enterprises for various reasons, including improved productivity, increased deployment frequency and improved security. While it’s true that GitOps continues to be a transformative trend, the increased speed and automation can also lead to misconfigurations that compromise application stability and security.

    Figure 2: CI/CD using GitOps

    Often, these security vulnerabilities delay development as developers spend significant time and effort investigating them. Since this poses a hindrance to progress, the question that arises is how enterprises can maintain the frequency and speed of deployment while remaining secure and reliable. This is where trusted delivery comes in.

    An Extension of DevOps

    DevOps is a software engineering practice and culture that blends development and operations to provide a shared and collaborative approach to developing and maintaining software. This can boost productivity, lower costs and increase customer value through increased collaboration.

    GitOps extends DevOps to bring the deployment workflow closer to developers. GitOps integrates code and configuration management for cloud-native application deployment and administration, while DevOps fosters shared ownership, transparency, and rapid feedback.

    GitOps uses DevOps-proven practices like version control, code review and CI/CD pipelines with any Git-based repository, including GitHub, GitLab or BitBucket. It builds on DevOps and applies DevOps best practices for infrastructure automation.

    What is Policy-as-Code?

    When deploying and managing cloud-native infrastructures, adhering to security best practices and regulatory standards is critical without hindering development. Policy-as-code is an approach to infrastructure security in which rules are codified and enforced, thereby reducing or eliminating human errors. You can use it to write applications using a high-level language to control, manage and automate policies. With policy-as-code in place, developers may create applications to manage security, compliance and other regulations seamlessly.

    Here are the types of policies that you can codify:

    • Security Policies: By adhering to these policies, you can protect your software from data breaches caused by misconfigurations.
    • Resilience Policies: The use of these policies in your Kubernetes clusters will help you create highly available, fault-tolerant systems as well as maintain business continuity.
    • Coding Standards Policies: These policies can enforce coding consistency, i.e., any code that violates these policies will be rejected automatically by the system.

    What is Trusted Application Delivery?

    Trusted delivery entails codifying security policies within your software delivery pipeline for security and compliance and introducing guardrails at every stage. By ensuring the security, integrity and reliability of your applications deployed in production environments, unauthorized access, tampering or malware introduction is minimized. It establishes and maintains trust in application source code, artifacts and deployment pipelines throughout the SDLC process.

    The key objectives of trusted application delivery are:

    • Security: It protects the application from security threats and vulnerabilities
    • Integrity: It verifies the authenticity and integrity of the source code of your application, together with the dependencies, and any artifacts
    • Reliability: It ensures that the applications deployed to the production are stable and are devoid of any security loopholes so that they can perform as desired

    Trusted application delivery helps development teams release applications early and secures them with automated protections or guardrails. It extends the GitOps pipeline by integrating security, governance and verification with policy-as-code. These security measures are implemented with policy-as-code. The policy-as-code approach encompasses the entire SDLC, including the design, development, testing and deployment stages.

    According to your organization’s security and compliance policies, compliance and security rules are codified and integrated into the system. Throughout the life cycle of your application, you can manage its security, compliance and other regulations using policy-as-code. These code-defined policies prohibit the provisioning of non-compliant resources. You should implement trusted application delivery practices to secure your organization’s application delivery process and reduce the likelihood of unauthorized access.

    Here are the key benefits of trusted delivery:

    • Faster and more frequent releases
    • Enhanced security
    • Protect against misconfigurations
    • Avoid compliance violations
    • Improved collaboration

    What is DevSecOps?

    DevSecOps is a software development methodology that blends development (Dev), security (Sec) and operations (Ops) at all phases of the SDLC by incorporating mandatory security checks. An organization of any size can benefit by creating a culture of shared responsibility and collaboration by bridging the gap between development, security and operations teams.

    A DevSecOps pipeline adds practices such as scanning, policy enforcement, static code analysis, threat intelligence and compliance validation to the SDLC process. Contrary to traditional software development processes that treat security as an afterthought leading to security vulnerabilities and breaches, DevSecOps is a paradigm shift that fosters a culture of proactive and continuous security approach to help address security concerns and vulnerabilities.

    DevSecOps and GitOps help organizations adapt quickly to changing emerging trends and requirements. DevSecOps is an offshoot of DevOps that focuses primarily on security throughout the development process. It helps detect and resolve security issues and vulnerabilities easily and quickly while fostering a culture of continuous improvement.

    Here are the benefits of DevSecOps at a glance:

    • At the outset, determine which points in the SDLC process are susceptible to security flaws and vulnerabilities in the SDLC process
    • Streamlined compliance
    • Secure supply chain
    • Reduced risk of the attack surface
    • Complete visibility of potential threats

    Why DevSecOps?

    DevSecOps is a proven way to secure your CI/CD pipelines, deliver secure, trusted, high-quality software and foster an environment where security is the responsibility of everyone in the organization. By shifting security left—i.e., introducing security into the CI/CD pipeline earlier—organizations can be much more productive and the teams can be more cooperative and collaborative.

    Given the threats to the security of the software supply chain, DevOps practices have given way to DevSecOps practices, where safety is paramount and a fundamental and ongoing component of software development. At the outset of the SDLC, you should ensure that software components and dependencies are secure to avoid potential software supply chain hazards.

    Additionally, it would help if you had built-in security gates in each step of the SDLC to ensure consistent, repeatable and automated processes from the software build phase to deployment. Remember, DevSecOps isn’t an option; instead, it is a necessity.

    Trusted Delivery With GitOps

    Automated software delivery pipelines are based on Git as the only source of truth. Policy-as-code is a great way to ensure that misconfigurations and errors are detected, alerted, and deployments halted automatically. Next, we’ll cover how a typical GitOps workflow determines where multiple checkpoints can be implemented for trusted delivery.

    In a typical workflow based on GitOps, a developer commits the source code to a Git repository. The CI tool then builds the container images. These container images are then tested and then uploaded to an image repository. At the end of this process, these container images will be deployed to a Kubernetes cluster. GitOps agents compare the current cluster state with the desired state on a continuous basis.

    Hence, you should implement security guardrails at the following points:

    • Git repository
    • CI stage
    • Configuration repository
    • Kubernetes cluster

    Creating a Trusted Software Supply Chain With DevSecOps

    You can build a trusted software supply chain by incorporating security practices throughout the SDLC to ensure the integrity and security of the software delivered by DevSecOps. Here is a step-by-step guide to building a trusted software supply chain by leveraging DevSecOps principles:

    • Security by Design: You should integrate security right from the outset of the SDLC and identify the key threats to the security of the pipeline.
    • Application Dependency Scanning: When building applications, we often use third-party dependencies to accelerate the development process. You should monitor security vulnerabilities and patches as they become available for these dependencies. You can leverage OWASP Dependency Check, Firewall from Sonatype, etc.
    • Secure Coding Practices: You should adhere to and enforce these practices among your development teams. You can integrate static code analysis tools into your DevSecOps pipeline, conduct regular code reviews and identify and fix security issues.
    • Container Security: You should scan container images regularly for security vulnerabilities, implement strategies to secure your containers and use container registries with access controls.
    • Continuous Compliance: You should ensure the software supply chain adheres to the relevant standards and guidelines for compliance requirements. In this regard, you should conduct audits regularly to check if compliance requirements have been satisfied.
    • Collaboration and Communication: You should be able to foster a culture in your organization that facilitates collaboration and communication across the development, operations and security teams.

    DevOps, DevSecOps and GitOps: The Future

    As organizations strive to optimize the efficiency of their software supply chain, DevOps, DevSecOps and GitOps are approaches that can improve collaboration, efficiency and security, to help software developers and IT operations succeed. These practices would become more critical and widely used in the future. GitOps provides teams with better tools for managing configurations so they can focus on more strategic activities.

    While GitOps is an approach that advocates Git-based version control systems as the single source of truth, DevOps is concerned about a cross-functional team that can work together. DevSecOps goes a step further by blending security into all stages of the SDLC process.

    The future of DevOps, DevSecOps and GitOps will see increased automation, tighter security integration and a sustained emphasis on efficiency and collaboration. Your organization should adopt these approaches to provide reliable software and services to its customers.

    Conclusion

    Trusted application delivery is essential for securing applications, protecting user data, meeting compliance requirements and preserving market credibility and trust. It helps enterprises and end consumers by ensuring that software is distributed securely and reliably. Organizations can ship high-quality applications with guaranteed security and performance by adopting trusted delivery practices.

  • Cycloid Adds Templates to Simplify GitOps Workflows

    Cycloid Adds Templates to Simplify GitOps Workflows

    Cycloid today added a Bootstrap Stacks capability that provides DevOps teams with the ability to build Git templates or use ones created by the company to define cloud computing environments.

    The capability is accessed via a portal added to the Cycloid platform for automating software delivery. It also makes it simpler for DevOps teams to reuse templates that are compatible with Terraform or Kubernetes environments as needed.

    In the months ahead, Cycloid plans to release Headers, a metadata tool that automatically populates project parameters in display pages and a project management tool to provide a unified view for each development environment, including continuous integration/continuous delivery (CI/CD) pipelines, documentation, automation, logs, cloud costs and carbon emissions.

    Cycloid founder Benjamin Brial said the templates provide a layer of abstraction that makes it simpler to implement GitOps workflows. As a best practice, GitOps enables DevOps teams to unify the management of code to provision IT infrastructure and software delivery. However, providing access to a Git repository alone is insufficient, said Brial. DevOps teams need access to a set of templates that automate the process and eliminate what is otherwise a complex set of tasks, he noted.

    Easily discovered templates also eliminate organizations’ tendency to create redundant automations simply because a development team was unable to find an existing framework that another team already created, added Brial.

    That approach also enables DevOps teams to more easily embrace platform engineering principles to centralize the management of DevOps using a Bootstrap Service catalog embedded within the Cycloid platform, said Brial. That capability makes it possible to enforce GitOps policies in a way the developers will have to respect because the automation frameworks are readily available, he added.

    It’s not clear whether organizations are embracing platform engineering. Historically, adoption of DevOps workflows in most organizations has tended to flow from the bottom up. As a result, it’s fairly common for multiple application development teams within an organization to have adopted multiple DevOps platforms to manage workflows. Advocates for platform engineering are making a case for eliminating many of those redundant platforms in favor of a corporate standard.

    At the core of any approach is the assumption that centralization will reduce costs by eliminating redundancy. Issues can arise, however, when centralization results in loss of flexibility. Many of the teams that adopt DevOps want to be able to add and replace tools as they see fit. If there is a corporate standard that narrows their tool options, it’s just a matter of time before those teams start to look for ways around the corporate standard.

    Each organization will need to strike a balance between flexibility and the need to bring some order to DevOps workflows. In the meantime, however, the pace at which multiple software development initiatives are being launched is likely to force a platform engineering conversation that, in many cases, many organizations have postponed longer than anyone cares to admit.

  • Coming Soon: AutoOps

    Coming Soon: AutoOps

    I’ve spent a lot of time looking at products in the DevOps toolchain and paying attention to where they are going, what they are doing that is new and how they are differentiated. That has led me to one inescapable conclusion: AutoOps is on the way.

    What is AutoOps?

    AutoOps is full robotic automation; the continuing progression of what has been building for the last couple of years. We have DevOps, GitOps, DevSecOps, IaC, CloudOps, AIOps … and the list is still growing. But they all move toward one thing: AutoOps.

    Now that development has been fully integrated into DevOps, the market is moving to do the same for operations. It is harder with operations because DevOps had Agile, but it is coming. And it is coming to your org.

    That is a good thing, in my opinion. Compliance almost demands it; what we currently call AI enables it and staffing in hard macroeconomic times will make it appealing.

    The questions that you should start asking now are what does the organization need to enable rapid deployment, management and upgrades while leaving humans in control for the majority of decisions?

    Some of you are living this. I have a friend that is in a GitOps shop with automated CloudOps where they are implementing AIOps to handle minor day-to-day things and monitoring for drift … but they’re the exception; most of us are nowhere near there.

    Knowing that is where we are heading means you can plan – and it is glaringly obvious this is where we are heading. Vendors are branching from their current subset of XOps to take us there and eventually, we will have a full end-to-end vendor that can do everything from GitOps to fully automated rollout, green/blue deployment, monitoring and drift correction with human intervention only where the organization wants it. Some vendors are already very close to this environment and I expect that several will get there in the next year or two via acquisitions or partnerships.

    We just don’t think of the overall DevOps/concept to retirement SDLC process in the absolute terms that AutoOps requires, but we’re going to want to because the environment is changing. The changes are for the better, but they will be massive and cause disruptions. However, they’ll make everything better in the end by eliminating error-producing bits while leaving us in control of the flow and process.

    Start Planning for AutoOps

    So start planning, understand what your organization wants and needs from the coming full-robotic automation I call AutoOps, and keep kicking rear. Eventually, we’ll have systems that can do a lot of Dev and Ops work, but that day is a ways off, and will require people to manage the systems that manage the systems… Funny but true.

    *I’ve never seen AutoOps used anywhere with regards to IT, I just thought it describes where we’re headed pretty well. Let me know if you’ve seen a better term coined by a smarter person. Or a not-smarter person that has a cooler name for it.

  • Forget Change, Embrace Stability

    Forget Change, Embrace Stability

    DevOps is about change. It was introduced to accelerate change, and it has opened up a whole new world of tools and possibilities that are all based on the premise of changing IT. Starting with Agile, we now have DevOps variants and subprocesses that run the entire software development life cycle (SDLC) process.

    Want to make more frequent updates that can regularly show demonstrable progress to business owners? Agile and CI! Need to get the app settled in its inevitable home with automation? You need CD! Want testing and zero-day protections? Start with DevSecOps built into the CI/CD process and add in application and API security! Want to fully automate deployment in the whay builds are automated? Grab a GitOps tool and let’s goooo! The list goes on and on. Literally. We’ve almost got more tools than we have names for spaces at this point.

    And all of those tools introduce change. Most of them introduce a massive amount of change. You don’t get to GitOps without breaking some of your existing processes and reworking them. We have fallen in love with change in IT.

    But there is a funny thing that’s happened along the way; I’ve mentioned it before, but never from this perspective. All great changes require time for consolidation. Be it stock market swings or revolutions, afterward, there is a period of consolidation. In IT, containers are a good example. When they became capable of actually running workloads, IT experienced a period of time when making all the various machines run in VMs was the thing. People weren’t (generally) looking for the new big change in the process, they were just glad that those one-off machines that had custom hardware could now be moved to something more manageable and that spinning up servers could be done in days or weeks instead of the months it used to take when you had to order hardware. But that consolidation created organizational standards and stability that the organization enjoyed for years – even after containers rose to challenge VMs and some organizations even today.

    So every once in a while, take a breather, consolidate gains, create standards that have meaning for your organization, then continue on with whatever the shiny new change is. Make sure that once you have identified, for example, the need for code scanners that can work within or that are complementary to the integrated development environment (IDE), that you have one scanner—or one scanner per programming language—to maintain and monitor instead of one tool per project. That saves time and creates stability, both of which will serve the organization moving forward.

    Change is here to stay; it’s the only constant. IT is caught in a cycle of change followed by change, with the next wave driven by generative AI. But in times of change, seek stability and, in times of stability, seek changes that can have a positive impact. So go standardize and organize before the next wave hits.

    Mostly, the changes will serve the organization by freeing up your time and letting you spend more of your time rocking the IT world. Those systems stay up because of you. More time is more innovation, and more stability means less burnout. All of which is good for the organization.

  • Start Automating Now

    Start Automating Now

    A bit of feedback I received from last week’s post is that I didn’t provide any background, I just jumped right in with examples. So let’s take a step back and talk about the bigger picture.

    It is always a surprise to me when I talk to enterprise friends and associates and hear that automation has not taken over their DevOps processes. Yes, we still (and might always) want the ability to have a person review what is being done and make course corrections if the automation has something wrong, but most of the automation that is available for DevOps is no longer new and should be a no-brainer.

    The ability for a build to block based upon any given criteria and notify Jira or ServiceNow—or whatever issue tracking system you use—while alerting the people responsible for the build is simply obvious … and yet, some organizations aren’t even that far yet. The ability to stop a build, create a ticket, assign it to the appropriate person and give it a priority is built into most of these products. It should be looked at to see if it’s suitable to the organization.

    This goes on and on. We’re now to the point where everything in the toolchain can be automated. IaC and GitOps are tackling the broad operations end, and if the application is 100% public cloud-based, CloudOps simply solves the problem. Automation from check-in to production is possible, with very few places where intervention from humans is needed.

    Given that most of this technology is mature—even though some of it has not been around as long but has broad enough adoption that it can be considered mature (Flux and Argo come to mind here)—it is time for enterprises to start another round of automation. Just like AI won’t eliminate most development jobs this year, automation won’t eliminate jobs, either. But we’ve been so strapped for time for so long that this may be a surprise. Automation might reduce the need to scramble for new team members while asking current employees to give a bit more … over and over.

    So get started. The math that made DevOps appealing in the first place—streamlining to reduce time to market and free up IT hours—is the same math that applies to the current round of automation. The more that can be automated, the more time team members will have to perform more productive work.

    Having mentioned AI in passing, I’ll use that as a segue to include AI in the “automation” discussion. At this time, nearly all of the tools we call AI are simply advanced automation tools and should be considered in the process of increasing IT automation. The ability for an app to run, learn your systems and improve them (be that improvement in development, security or ops) is just advanced automation. In the places where it works well in a given environment, it should be adopted. It is highly unlikely that AI will get worse over time, so even marginally acceptable AI results make it worth implementing, counting on technology advancements and continued training to improve that benefit moving forward.

    So, start automating now. Free your time, take a holiday at the beach, do the same job in less time or, to quote a previous generation, “Work smarter, not harder.” You’re doing the job that keeps the servers humming and users coming, now do it with less effort. And keep rocking it.

  • Declarative Compliance With Policy-as-Code and GitOps

    Declarative Compliance With Policy-as-Code and GitOps

    In recent times, the quest for greater agility, faster releases, enhanced scalability, security and performance brought forth the advent of several automation tools, technologies and frameworks. Software development has evolved considerably over the years to mitigate these challenges.

    Monoliths have been split into microservices for improved scalability, maintenance and faster releases. Since DevOps is adept at bridging the gap between development and operations teams, now these two teams can work together to improve productivity and speed up release cycles.

    The Need for Compliance

    One of the critical strategies for business success is to stay compliant since processes are always improving and evolving. Compliance ensures that your DevOps processes and practices are consistent and meet regulations or regulatory standards.

    An organization with a compliance-driven DevOps culture can cut down on operational costs, improve efficiency and reduce risks considerably.

    What is Declarative Compliance?

    Declarative compliance is defined as the process of managing and enforcing compliance in which policies are defined using code, and the system’s desired state is declared rather than explicitly programmed. To enforce compliance policies, you don’t write scripts or programs; instead, you use a high-level language or configuration file.

    Declarative Compliance provides several benefits:

    • Increased efficiency
    • Better managed compliance
    • Minimized risk of human errors
    • Enhanced security
    • Faster deployment
    • Better collaboration

    With declarative compliance tools, the system’s state is defined as a set of policies, and the tool makes sure it matches the desired state. The tool will automatically fix anything out of compliance, bringing the system back into compliance if there’s a deviation.

    Tools for Declarative Compliance

    Some of the popular declarative compliance tools are:

    • Open Policy Agent (OPA)
    • Terraform
    • Kubernetes Policy Controller
    • Chef Compliance
    • AWS Config

    Declarative Compliance for Infrastructure and Application Deployment

    Often, declarative compliance is used in infrastructure management and application deployment, where policies are written as code and enforced across multiple environments. In addition to managing compliance at scale, declarative compliance tools ensure that the system’s desired state stays consistent across multiple systems and applications over time.

    With declarative compliance, organizations can manage compliance more efficiently, effectively and securely since they can define policies as code and enforce them automatically. In addition to reducing errors, downtime and security breaches, this approach enables organizations to stay compliant and maintain their system consistently.

    Introduction to Policy-as-Code

    A policy-as-code approach allows you to write programs that govern security, compliance and rules throughout your application’s life cycle. It entails defining and applying code to manage and automate policies. It shares the same ideas as infrastructure-as-code (IaC) and other DevOps practices like continuous integration and continuous delivery (CI/CD).

    A policy enforces constraints and restrictions to prevent unauthorized access to resources such as databases, storage, services, etc. By codifying policies, you can enforce rules and constraints throughout the development life cycle to prevent non-conforming resources from being deployed by integrating these policies into the software development life cycle.

    compliance policy-as-code

    Figure 1: Policy-as-Code at work!

    Why Policy-as-Code?

    Typically, businesses have the required processes, tools and procedures to guarantee the deliverable’s quality. Performance, scalability, ease of use and accuracy are typical quantifiable quality metrics used to ascertain the quality.

    You can examine the quality of your application using these metrics, but a manual procedure will take time and is prone to mistakes. A better alternative is to leverage policy-as-code.

    The traditional policy enforcement processes have certain drawbacks. It is challenging to perceive that your policies would be entirely adhered to, i.e., they will not be broken. It is cumbersome to check against a list of policies manually.

    Moreover, the traditional policy enforcement method cannot scale in today’s agile world, where organizations have been growing continuously with more and more employees, services and teams.

    Here are the key benefits of policy-as-code at a glance:

    • Faster and more efficient deployments
    • Automated governance, approvals and policy enforcement
    • More reliable and secure applications
    • Centralized policy management
    • Leverages version control
    • Better visibility of policies
    • Enhanced collaboration

    How Does it Work?

    Policy-as-code involves three key elements:

    1. The policy itself, which contains the required code to model the decision-making process
    2. The data and information about the environment, service or application
    3. The query that is responsible for triggering the decision-making process

    Here’s how policy-as-code works:

    1. Policies are defined as code in a human-readable format
    2. These policies are then translated into machine-readable code using Rego or HCL
    3. The policies are integrated with the IaC pipeline
    4. A policy engine evaluates the infrastructure and the application resources against these policies
    5. The policy engine accepts the policies as input, processes them and then produces a query result
    6. The policy engine reports violations (if any) to the respective stakeholders, such as the development or security teams
    7. If violations are reported, the necessary remediation is performed automatically or manually

    Types of Policies

    • Security policies: Security is enforcing adequate technological controls to safeguard company assets.
    • Compliance policies: Compliance policies are designed to help organizations adhere to governance standards. Typical examples include PCI DSS, GDPR, HIPPA, etc.
    • Operational excellence: This encompasses policies pertaining to service degradation or interruption.

    What is GitOps? How can GitOps Help With Compliance?

    GitOps is a subset or extension of DevOps encompassing a collection of practices that can simplify deployments using Git. It is an operating model for cloud-native applications that leverages Git as a single source of truth for declarative applications and infrastructure. Using GitOps, you can implement a standardized policy-as-code approach across your CI/CD and GitOps pipelines.

    The key principles of GitOps are:

    • The state of the system is captured and stored in a Git repository
    • The entire system is configured declaratively
    • Automated deployment

    You can create policies in Git and store them in a particular repository, which is essential since Git offers sophisticated change management tools like version control. Only the compliance team would have admin access to the repository.

    It’s easy to enforce policies with Git, and you can apply them to any execution engine, like CI or CD. With GitOps, you can audit and document every action that affects data consumption.

    You can expose system changes, optimize deployment and use a version-controlled infrastructure. By managing compliance with GitOps, you can make sure that the changes are visible, verifiable and auditable.

    GitOps

    Figure 2: GitOps at work!

    Using GitOps as the Execution Engine in a GitOps Process

    There are several benefits to using GitOps as a process execution engine, including:

    • Increased automation: GitOps automates software delivery, reducing manual intervention and increasing deployment accuracy and speed.
    • Better transparency: GitOps records all changes to the system in a transparent and auditable way so that you can track compliance easily.
    • Maintaining consistency: GitOps keeps the system in the desired state defined in the Git repository, reducing non-compliance risk.
    • Fosters collaboration: All changes to the system are made through the Git repository, so GitOps encourages collaboration across teams.

    Using Policy-as-Code and GitOps: Policy-Based Compliance

    By using GitOps and policy-as-code, teams can manage compliance-as-code in an auditable, declarative way and enforce it via GitOps, ensuring that the desired state is always maintained. This can help your teams to codify compliance requirements as code, store them in version control and automatically enforce them through GitOps.

    Organizations can improve the consistency and reliability of their systems by defining and enforcing compliance requirements using policy-as-code and managing their deployment processes with GitOps. While this can minimize compliance violations, it can also maximize efficiency.

    By following policy-based compliance declaratively, businesses can define policies as code and enforce them automatically and repeatedly. By adopting such an approach to compliance management, you can better manage compliance in complex systems in a consistent, scalable, agile, transparent and collaborative manner.

    For example, you can leverage policy-as-code to define policies for infrastructure provisioning and configuration management to ensure that the infrastructure complies with organizational policies and industry standards. Typical examples are defining policies for network configuration, access control and resource allocation.

    Use Cases: Using Policy-as-Code and GitOps in a Production Environment

    Infrastructure Provisioning

    Open Policy Agent (OPA) can define policies for infrastructure resources like AWS Elastic Compute Cloud (EC2) servers. You can use GitOps to manage the deployment of infrastructure resources by storing IaC files in a Git directory. When the IaC files are changed, GitOps is adept at automatically deploying the changes while OPA checks for policy compliance.

    Deployment

    You can use policy-as-code tools such as OPA to create application deployment policies requiring specific security settings, while GitOps manages the application deployment by storing manifests.

    Manage Kubernetes Clusters

    You can leverage policy-as-code to better manage your Kubernetes clusters. So, you can write your code to define policies to manage Kubernetes resources, such as nodes, pods, clusters, etc., declaratively.

    Compliance Management

    You can use policy-as-code to establish and implement rules and regulations, such as those pertaining to security, to help businesses ensure that their systems comply with industry regulations and standards.

    OPA can be used to set compliance policies, like restricting access to sensitive data to authorized users only. You can use GitOps to manage the deployment of compliance policies seamlessly.

    Trusted Delivery: Policy-as-Code Integrated With GitOps Workflows

    Trusted application delivery is a software delivery approach emphasizing security and trust in the software supply chain. This ensures that any software delivered to the clients is devoid of security flaws, malicious code and other attacks.

    This approach prioritizes security and trust at the development, testing, packaging and deployment stages and integrates security and compliance checks into the GitOps pipelines. This quality check reassures clients about the product’s quality and ensures that any modifications adhere to predetermined rules and principles.

    Trusted application delivery allows for quick deployment while securing apps with automated guardrails. These guardrails are implemented using policy-as-code. By enforcing these guardrails, you encourage frequent deployments while ensuring that the reliability and security of the application are not compromised.

    Conclusion

    Declarative compliance with policy-as-code and GitOps is a best practice for achieving greater compliance, reliability and agility in software development and deployment processes. You can simplify defining and enforcing compliance policies using declarative configuration files and automation tools. Using Git as the single source of truth for all configuration changes, an organization can achieve greater efficiency, visibility and auditability in the compliance process.

  • Back to Basics

    Back to Basics

    We here at devopsy.kinsta.cloud do sometimes suffer from The Curse of Knowledge. We write about DevOps regularly for an audience that is doing or managing DevOps. We assume knowledge, but there are still a fair number of people and organizations who are unclear about some of the terminology/technology that we talk about.

    So if you are a regular reader of my blog, this might be a redundant post—but read on because it might have info you haven’t had a reason to consider yet.

    I’m going to start with the obvious—how most readers got here to begin with: DevOps. DevOps is automating the build process so that builds are automated and repeatable. Teamed with Agile development, DevOps means that development teams can rapidly iterate through development and generate regular updates to their applications. Everything else in this blog is based on the premise that it is running in a DevOps environment.

    Continuous integration: Typified by Jenkins, continuous integration (CI) is the part of DevOps that runs the automated builds. It includes reporting and defect resolution only in the sense that it feeds into other systems that track defects. With CI in place, a developer or DevOps engineer can say, “Okay, ready to test a build,” and with one command, can kick off a build that will result in a finished artifact. Or result in a built product, anyway. ‘Finished’ is optional, and the target of CI is not generally a running application.

    Continuous delivery: One of the most confusing things for new DevOps people is CD. The way that the space evolved, there are actually two things that CD could stand for; continuous delivery–automated build of the complete application —or continuous deployment—delivery of a running application onto infrastructure. Continuous delivery is straightforward and means all the bits of the application are code complete–or at least the current state of the application is complete enough to build an application. Continuous deployment is much more convoluted. It might mean deployed to a development environment for developer-level testing or a test environment for running automated (continuous testing) tests on or in production. It might be a staged rollout, where users are siphoned over, or it might be a hard rollout where the old version is replaced by the new in a manner similar to ripping a Band-Aid off.

    Continuous test: Automated testing is at the heart of continuous test (CT), and it is only now becoming a real possibility for most short-staffed enterprises. Advancements in machine learning (ML) mean that tests can be run, results filtered to eliminate false or duplicate results, and IT staff is only notified of the important bits. Unlike traditional testing, staff is required only to create the tests and to look at meaningful results.

    Infrastructure-as-Code: Infrastructure-as-Code (IaC) is the ability to define all relevant elements of infrastructure with text files. Often this is done in YAML and sometimes in a variety of other formats. The idea is that infrastructure provisioning can be on-demand and adaptable. It is certainly more agile to change a text file from port 80 to 8080 and then run a command to process the text file–now the port is documented and whenever needed the command can be re-run to move to a known state. IaC largely relies upon software implementations over physical. While the idea of software-defined networking (SDN) has fallen out of favor, the underlying principles are everywhere and part of IaC, for example.

    GitOps: All the rage these days, GitOps is probably best defined as applying DevOps to operations. That does sound weird, but I like to break it up as “traditional DevOps is DevOps, while IaC (to a lesser extent) and GitOps (to a huge extent) are DevOps.” The point being that GitOps uses the text files generated in IaC and worker nodes generally designed for continuous deployment and then performs operations tasks based on those text files. If IaC defines the entire environment that an application needs and DevOps defines all of the sources that an application is composed of, bundling the two together will enable full-on automation from conception to decommissioning. When a new version of a source file or configuration file is checked into Git, the system then makes certain that every instance of the application is updated with the new version. Of course, this process, like every one listed here, is far more complex than that in implementation; we are talking about ‘what, not how’ at this level.

    All of this really is the very tip of the iceberg, but that was the goal here – to offer readers a place to start if they’re finding themselves looking at DevOps for the first time. As you dig deeper, things like DevSecOps and test driven development (TDD), or defect tracking and Agile methodologies start to play into the landscape, and maybe I’ll write an intro bit for those also. But this time, we’re out of space.

    As always, keep rocking it. No matter which or how much of the above you are using, if the organization’s applications are up and running, that’s you. And you deserve more kudos than you are likely getting. But keep grinding through—the rest of us get it, and thank you.

    Image source: Photo by Towfiqu barbhuiya on Unsplash

  • The Differences Between DevOps and GitOps

    The Differences Between DevOps and GitOps

    The transition to a DevOps model has been welcomed by many organizations that are looking to implement more efficient development processes. However, there are multiple models to choose from, with GitOps a popular choice for many. So what is the difference, and which one is better?

    In this article, we will discuss what each model entails and their key differences and establish if one is better than the other for your business.

    What is DevOps? – An Overview

    The DevOps model involves several tools and practices designed to make software development much more efficient. This increase in efficiency is achieved by implementing more automation, adopting continuous delivery practices and improving the communication channels between the developers and operations. 

    Ultimately, DevOps is designed to assist organizations in deploying new products and systems faster. By establishing more collaboration between development and operational teams, changes can be made quickly to best suit the business’s overall goals. 

    There is no single tool that is required for DevOps. In fact, your team can use any tool that effectively improves communication and automates key tasks. 

    How to Integrate a DevOps Strategy

    Adopting and integrating a DevOps strategy into your organization’s overall workflow is not a difficult process, and it can be achieved by following these steps: 

    1. Create a continuous integration/continuous delivery (CI/CD) pipeline.
    2. Configure this new pipeline to enable the automated deployment of changes to your development/production environment.
    3. Set up an alert and monitoring system for your development environment. 
    4. Create collaboration channels so operational teams can inform the developers of new opportunities to improve the business’s workflow. 

    An Overview of GitOps

    GitOps is a modern concept that was initially introduced by the Kubernetes management system Weaveworks. This model uses Git, an open source, distributed version control system, to track any changes to a system and who has made them. 

    The GitOps model uses a Git repository as the single source of truth (SSoT), improving collaboration between developers and operational team members. This, in turn, allows organizations to automate the deployment process. 

    By leveraging CI/CD pipelines like DevOps, changes are automatically applied to the codebase without any need for manual input. Thanks to Git, these changes can also be easily reversed if needed, reverting to a previous working version. 

    How to Integrate a GitOps Strategy

    GitOps can be integrated into a business’s workflow quite simply by following four key steps: 

    1. Set up a Git repository for your organization’s codebase. When integrating a GitOps strategy, an organization does not need to use GitHub exclusively, as any code repository that uses Git commands can do the job. 
    2. Connect this repository to a CI/CD tool.
    3. Configure your chosen tool to deploy automatic changes to your development environment. Popular tools include Jenkins and Circle CI. 
    4. Set up monitoring and alerts for your development environment. 

    It should be noted that the CI part of GitOps still needs to go through your DevOps tool, whether it’s GitHub Actions or AWS CodeBuild. 

    The Differences Between DevOps and GitOps

    The core difference between the two is that GitOps allows teams within an organization to manage and deploy changes to infrastructure using Git as the single source of truth. DevOps, on the other hand, focuses on effectively connecting development and operational teams using a tool of the organization’s choosing. 

    Either model can offer a number of benefits in terms of development and deployment, but there are notable differences between the two that should be considered. It should also be noted that DevOps and GitOps can work in unison as long as continuous integration (CI) is delivered using your chosen DevOps tool.

    DevOps – Key Elements

    • The DevOps model is considered a working culture.
    • DevOps can use a variety of CI/CD pipelines and is not restricted to any one tool.
    • This model increases automation and the regularity of deployment, reducing any human input. 
    • Cloud configuration can be used for coding, supply chain management and other key requirements.
    • The pipeline separates development and operations processes while improving collaboration between the two.

    GitOps – Key Elements

    • GitOps is a development and deployment technique that relies on the Git version control system.
    • Git is used as the single source of truth for all changes to the codebase.
    • GitOps can use Kubernetes and infrastructure-as-code (IaC) practices and various other CI/CD pipelines. 
    • The GitOps model is designed to increase development times and reduce reliance on lengthy scripts.
    • It also enables a smoother transition between a project’s development and operation stages.

    DevOps Vs. GitOps – Which Model Is Better?

    Both GitOps and DevOps are becoming increasingly popular when it comes to adopting new approaches to improve software development processes and making deployment quicker and more efficient. As outlined above, the difference between the two models largely comes down to the tools that are used, but the result is roughly the same in terms of the overall benefits. 

    Deciding which approach is the right one for your organization or determining which one is “better” will always come down to the needs and requirements of your organization. In fact, the ideal scenario would be to integrate GitOps so it works alongside your DevOps approach. 

    GitOps is not a replacement for DevOps, as the CI process still relies on DevOps tools. However, GitOps can improve the effectiveness of the DevOps team thanks to continuous delivery processes. Using Kubernetes-native mechanisms and commands and sticking to standard source code principles, GitOps does not require developers to change how they work, guaranteeing a seamless implementation process.

    Wrapping Up

    DevOps is a working culture that improves collaboration between development and operations teams and utilizes automation tools, resulting in rapid and more efficient software development. GitOps, on the other hand, uses Git as the single source of truth, making it easy to track changes and further increase development times thanks to continuous delivery processes. 

    Although each model can be used exclusively, they can also work alongside each other to deliver rapid software development and high levels of collaboration. Therefore, it is unfair to claim that one model is better than the other, as they can both play their part in an overall DevOps strategy.