devopsy.kinsta.cloud

Tag: infrastructure as code

Security, Automation and Developer Experience: The Top DevOps Trends of 2024

If you ask most folks to describe the top DevOps trends in 2024, you’ll likely hear buzzwords like AI or DevSecOps.

Those are certainly trendy topics. But based on my everyday work helping businesses plan and execute DevOps strategies, I’m noticing a different set of salient trends in DevOps. Although much is being said about how technologies like AI might impact DevOps, the biggest changes I’m seeing right now involve other types of solutions and techniques.

Here are what I view as the three most important current DevOps trends, along with a breakdown of how they may change DevOps tools and processes.

Policy-Based Management and IaC Drive DevOps Security Innovation

Security has always been a priority for DevOps practitioners. But right now, I’m seeing DevOps teams adopt new strategies to improve the security of the applications and environments they support.

One large-scale change is the greater use of cloud-based policy management to enforce security best practices in cloud environments. Teams configure cloud services and resources using the code-based configuration frameworks that cloud providers support, then scan the configurations to detect risks.

This approach makes it possible to enforce cloud governance consistently, centrally and automatically. Instead of simply writing governance policies and hoping that engineers remember to follow them when they configure cloud resources, businesses are building automated governance guardrails via policy-based management.

In a similar vein, more and more of the DevOps teams I work with are embracing static code analysis of infrastructure-as-code (IaC) templates as a means of detecting risks. As with policy-based management of cloud resources, an IaC-centric approach to infrastructure provisioning makes it possible to automate infrastructure management and identify security risks earlier in the development lifecycle.

Some teams use cloud policy and IaC code scanning to warn one another about security policy changes that might cause an application or service to break. They do this by inserting configuration changes into their code using “audit” mode if their configuration framework supports it, or by configuring IaC scanners to flag changes if an audit feature is not available. This allows engineers to detect whether a change might cause a problem for an existing deployment.

This is important because security and application teams operate separately in many organizations. When the two groups lack an efficient way to communicate about changes, they can disrupt each other’s operations – the “right hand/left hand” problem. Configuration scanning provides a systematic way to ensure that each group is on the same page when it comes time to introduce changes – and the “audit” mode approach provides a grace period that allows time to react before a change takes effect.

Doubling Down on DevOps Automation with GitOps

Another overarching trend currently reshaping DevOps is using GitOps to make DevOps automation more efficient and consistent.

GitOps is the use of Git (or a similar source control system) to manage automated DevOps workflows. It involves defining configurations using code and applying them through features like GitHub actions.

When you opt for GitOps, you move DevOps automation controls from individual workstations to centralized source control repositories. The result is the ability to track and manage all automated workflows via a central hub. Doing so increases efficiency and mitigates issues like different engineers working with different versions of the same automation frameworks on their personal desktops.

In addition, GitOps automatically generates records of what changed and how it changed, because every action is logged through the source control system. This isn’t exactly documentation in the traditional sense and means that GitOps comprehensively documents every change – which is beneficial because human engineers tend not to be so thorough when documenting their actions.

To be sure, GitOps is not without its challenges. Implementing GitOps effectively requires additional skills – namely, expertise with both IaC frameworks and source control systems – that not all DevOps engineers possess. I also notice a tendency on the part of some teams to set up GitOps pipelines but rely on manual approvals instead of automated actions to trigger changes. That approach largely undercuts the value of automating pipelines in the first place.

However, these are challenges that teams can solve through education and by fully leaning into GitOps. In addition, techniques like automated testing of GitOps configuration code can help to build teams’ confidence in automations and reduce reliance on manual approvals.

Going forward, expect to see more adoption of GitOps techniques among teams seeking to level up their approach to DevOps automation. Automating individual DevOps processes (such as software testing and deployment) won’t be enough; truly efficient organizations will turn to GitOps to automate their entire DevOps workflows, from end to end.

Investing in Developer Experience

Making software delivery processes more predictable and efficient is merely a step toward DevOps’s ultimate goal, which is to help developers become productive and satisfied with their jobs.

To that end, I’m noticing interest and investment in developer experience right now. This is playing out through two interrelated initiatives.

One initiative is platform engineering. It involves creating DevOps teams that specialize in certain functions, such as network management or security. Those teams are designated to support those functions throughout the organization. This approach reduces cognitive overhead for developers by freeing them from the need to handle types of work outside their main focus. In other words, instead of forcing developers to be DevOps generalists, platform engineering lets different teams focus on doing what they know and enjoy best – leading to greater productivity and higher levels of job satisfaction.

The other major developer experience trend currently playing out is developer self-service. This means developers can obtain the technical solutions they need on demand without a complicated procurement process. In most cases, organizations enable self-service by implementing internal development platforms (sometimes called internal developer portals) (IDPs) which host ready-made infrastructure resources and software environments that developers can deploy on a self-service basis.

There are risks inherent in these trends. They require specialized types of skills. When poorly implemented, platform engineering and IDP solutions can create more problems than they solve. However, when you ensure that your teams have the requisite expertise, and deploy a carefully planned IDP that gives developers access to the resources they need, you’re likely to see a significant reduction in friction within your organization and a boost in developer productivity and happiness.

Real-World Changes, Beyond the Hype Cycle

Admittedly, discussing DevOps trends centered on security, automation and developer experience may not be as exciting as debating whether AI will take away DevOps engineers’ jobs. But if you want to know what’s truly changing in the world of DevOps – as opposed to which conversations are most hype-worthy – these are the places to look.

Security, automation and developer experience are also among the domains of DevOps where there is a great deal of opportunity to innovate – and, indeed, where adopting new tools and techniques will be critical for organizations that don’t want to be left behind as DevOps evolves.

May 17, 2024
The Basics of Infrastructure as Code: A Primer for DevOps
Cloud computing and DevOps are reshaping how organizations design, build and manage their IT infrastructure and applications. Central to this transformation is the concept of infrastructure as code (IaC).

IaC changes the way IT operations and development teams collaborate to create, provision and manage infrastructure and applications. The practice treats infrastructure as software, enabling automated provisioning, configuration and management.

The process needed to change. Traditionally, managing IT infrastructure was a labor-intensive and error-prone process. System administrators manually configured and maintained servers, networks and storage systems, which led to inconsistency and operational bottlenecks. As organizations adopted cloud computing, they were motivated to address their needs for agility and automation.

At its core, IaC involves defining infrastructure and application components using code, which can then be version-controlled, tested and deployed in the same manner as software applications. IaC emphasizes the use of declarative or imperative code to define infrastructure, which allows for the automation of tasks that were once manual and time-consuming.

Among its benefits:
- Automation and Consistency: IaC allows organizations to automate the provisioning and management of infrastructure. The result is that configurations stay consistent across environments, reducing the risk of configuration drift and human errors.
- Version control: Just like software code, IaC scripts can be stored in version control systems. This enables tracking changes over time, rolling back to previous configurations and team member collaboration.
- Scalability: IaC makes it easier to scale infrastructure up or down based on demand. It’s easier to allocate and deallocate resources dynamically.
- Speed: The time required to provision and configure infrastructure is significantly reduced. This agility allows organizations to respond quickly to changing business requirements.
- Reusability: IaC encourages the creation of reusable templates and modules, which reduces duplication of effort.
- Testing and validation: Infrastructure code can be tested automatically, ensuring that configurations are correct and meet security and compliance standards before deployment.
IaC in Cloud Computing

Cloud computing platforms such as Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) have played a pivotal role in popularizing IaC. These platforms provide a wide range of services and APIs that can be orchestrated and managed through code.

For example, Netflix, the global streaming giant, relies heavily on cloud infrastructure to deliver content to millions of viewers worldwide. It embraced a microservices architecture and employs a tool called “Spinnaker” for continuous delivery. Spinnaker uses IaC to define and manage the infrastructure required to deploy and scale microservices. Netflix engineers write code to describe an application’s infrastructure, enabling them to create, modify and tear down resources on demand. This approach has allowed Netflix to achieve rapid deployment, reduce downtime, and maintain high availability while managing a complex and dynamic infrastructure.

As another example, Shopify, the e-commerce platform that powers thousands of online stores, uses IaC to manage its infrastructure across multiple cloud providers. Shopify uses Terraform, a popular IaC tool, to define and provision cloud resources. With Terraform, Shopify can define its infrastructure clearly and concisely, ensuring consistency and repeatability. Everything is defined in code, whether the Shopify staff aims to create a new instance, configure a load balancer, or scale an application. This approach has enabled Shopify to scale rapidly, maintain stability and adapt to changing customer demands seamlessly.

IaC in DevOps

DevOps aligns perfectly with IaC:
- Continuous Integration/Continuous Delivery (CI/CD): IaC is a cornerstone of CI/CD pipelines. In a DevOps environment, changes to infrastructure and applications are continuously integrated and tested. IaC scripts define the desired state of infrastructure, ensuring its consistency across all stages of development.
- Collaboration: Developers can define infrastructure requirements alongside application code, leading to a shared understanding of infrastructure needs and reducing the “it works on my machine” problem.
- Immutable Infrastructure: IaC promotes the concept of immutable infrastructure, where servers and resources are not modified in place but replaced with new instances when changes are needed. This approach ensures consistency and simplifies rollback procedures.
- Infrastructure Testing: Infrastructure as code can be subjected to automated testing, including security scanning and compliance checks. This approach helps identify and rectify issues before they reach production.
- Feedback Loop: Infrastructure issues or discrepancies are detected early in the development process, reducing the cost and effort required to address them.
As technology continues to evolve, the role of IaC in cloud computing and DevOps will only become more prominent. Organizations that adopt IaC stand to gain a competitive advantage by accelerating their development cycles, reducing operational overhead and improving infrastructure reliability.

The adoption of infrastructure as code is not just a trend but a necessity in the age of cloud and DevOps. It empowers organizations to build and manage infrastructure with the same rigor and discipline as software development, paving the way for a more agile and efficient future in IT operations.
May 10, 2024
The Role of DevOps in Orchestrating Enterprise-Wide Cloud Security

Moving to the cloud presents several security challenges for enterprises, including protecting the organization from data breaches, misconfigurations, compliance issues and unauthorized access. Moreover, the dynamic nature of cloud environments, with resources being provisioned and de-provisioned on-demand, adds complexity to security management. Organizations should implement robust security measures such as encryption, access controls and regular security audits. Training employees on best practices for cloud security is crucial to mitigate risks and ensure data protection in the cloud environment.

These items should be top of mind.

Automation: The practice of DevOps promotes automation across the software development lifecycle, including security processes. By automating security tasks such as vulnerability scanning, code analysis and configuration management, DevOps teams can identify and address security issues early in the development process, reducing the risk of vulnerabilities in production environments. Furthermore, incorporating security into the DevOps pipeline allows for continuous monitoring and testing of security controls, ensuring that any vulnerabilities are quickly identified and remediated. This proactive approach to security helps to strengthen the overall security posture of cloud environments.

Continuous Monitoring: DevOps encourages continuous monitoring of cloud infrastructure and applications. Through monitoring tools and techniques, DevOps teams can detect security incidents in real-time, allowing for immediate response and remediation. This proactive approach helps mitigate potential security threats before they escalate into significant breaches. By continuously monitoring for any anomalies or suspicious activities, DevOps teams can also improve their incident response capabilities and minimize the impact of security incidents. This ongoing vigilance is essential in today’s dynamic threat landscape to protect sensitive data and maintain the trust of customers.

Infrastructure as Code: With the adoption of infrastructure as code (IaC), DevOps teams can define and manage cloud infrastructure using code. This enables consistent and repeatable deployments and enhances security by ensuring infrastructure configurations adhere to security best practices. By treating infrastructure as code, organizations can implement security controls such as encryption, access controls and network segmentation programmatically, reducing the risk of human error and misconfigurations.

Collaboration and Communication: DevOps fosters collaboration and communication between development, operations and security teams. By breaking down silos and encouraging cross-functional collaboration, DevOps ensures that security considerations are integrated into every stage of the software delivery pipeline. This collaborative approach helps align security objectives with business goals, enabling organizations to achieve a balance between innovation and risk management.

Security First: DevOps promotes an approach where security is integrated into the software development process from the outset. By embedding security practices such as secure coding principles, threat modeling, and security testing into the development workflow, DevOps teams can identify and address security vulnerabilities early in the development lifecycle, reducing the likelihood of security incidents in production. This proactive approach helps organizations save time and resources by preventing costly security breaches that could occur later in the software development lifecycle. Collaboration between development and security teams means DevOps can enhance overall organizational resilience to cyberthreats by fostering a culture of security awareness.

Compliance and Governance: DevOps helps organizations maintain compliance and governance standards in the cloud. By automating compliance checks and implementing security controls as code, DevOps teams can ensure that cloud environments adhere to regulatory requirements and internal policies. This reduces the burden of compliance management and enhances security posture. Furthermore, by continuously monitoring and auditing cloud environments, DevOps teams can quickly identify and remediate any compliance issues that may arise. This proactive approach to compliance and governance helps organizations avoid costly fines and penalties associated with non-compliance.

DevOps is critical in orchestrating enterprise-wide cloud security by promoting automation, continuous monitoring, infrastructure as code (IaC), collaboration, communication, shift-left security, and compliance and governance. By integrating security into every aspect of the software delivery pipeline, DevOps enables organizations to embrace the agility and scalability of the cloud while effectively managing security risks. As enterprises continue to use cloud technologies to drive innovation and growth, DevOps will remain essential in ensuring that security remains a top priority. By implementing DevSecOps practices, organizations can proactively address security concerns early in development, reducing the likelihood of vulnerabilities in production environments. This holistic approach to security enhances overall cybersecurity posture and fosters a culture of shared responsibility and accountability across teams.

May 8, 2024
Mind the Shift: Answering Common Questions About OpenTofu
I recently returned from KubeCon EU where I participated in the first-ever OpenTofu Day and an Evolution of Infrastructure-as-Code panel discussion. During the conference, dozens of engineers and tech execs asked me questions about the OpenTofu project. They wanted to ascertain if it made sense for them to switch to OpenTofu, whether for testing purposes or to make the IaC management tool a permanent fixture in their tech stack.

Many people shared similar questions and concerns which, notably, were still the same as those expressed during the GA release. Now, on the heels of the OpenTofu v1.7 release, I’m taking the time to summarize my answers to those frequently asked questions and to share my (understandably biased) insights on the advantages of adopting it.

What Does “Drop-In” Replacement Mean?

In our communications, we commonly refer to OpenTofu as a “drop-in replacement” for Terraform. Unsurprisingly, the most common question I was asked was, “What does ‘a drop-in replacement’ really mean?”

As a developer, I understand where such questions come from. “Drop-in replacement” sounds more like a marketing promise than a technical specification. With that in mind, let me clarify what we mean.

To try OpenTofu, follow these steps:
1. Install OpenTofu.
2. Use the tofu command to execute your previously terraform commands (e.g.,tofu plan instead of terraform plan, tofu apply instead of terraform apply, and so forth).
And that’s it.

You don’t need to change a single line of code in your Terraform configuration. You can continue using all your favorite providers and modules, public or private. They will all just work.

By “drop in” we mean literally the minimal possible amount of effort with the fewest possible adjustments.

What’s the Best Time to Make the Switch?

Another question I encountered, often implied rather than asked directly, was, “Should I start with OpenTofu now or wait a while?”

Setting aside my personal bias, I can sum up my answer with a simple, undeniable fact: No future scenario is likely to make the switch easier than it is right now.

Let me explain.

The ease of transitioning between OpenTofu and Terraform comes from their complete interoperability in 1.6 stable versions. The ability to switch between Terraform and its open-source successor with a simple command alias means there’s effectively no cost to trying OpenTofu.

To me, this suggests that if you’re considering giving OpenTofu a try—whether sooner or later—the best time to do so is now.

Will the Projects Diverge?

Inevitably they will. However, there’s more to consider.

Ease of migration is a top priority for us on the OpenTofu team, and it is equally important to our community. This commitment ensures that we will manage any divergence between versions carefully.

Take OpenTofu v1.7 as an example. This version introduces several features, such as the much-anticipated state file encryption. But importantly, all these additions are opt-in. This approach exemplifies our commitment to not locking users into OpenTofu against their preferences.

Starting with OpenTofu is essentially a low-effort trial. The worst-case scenario is that you end up with a free, open-source version of Terraform that includes some additional, optional features to explore.

This was the mindset of several people I spoke with at KubeCon who were adopting OpenTofu as a future-proofing strategy. Doing so would let them keep their options open and help them avoid being locked into any specific vendor-ecosystem or IaC management tool.

What About Community Support?

Support for the project is gaining momentum. Key players including Alpine, Brew, and GitLab have already moved away from BSL-licensed Terraform binaries to ensure long-term open-source interoperability and to address their unique needs.

These are just a few examples. The shift in HashiCorp’s licensing terms, compounded by its recent acquisition by IBM, has sparked concerns about long-term viability and compatibility. These are unlikely to go away anytime soon. For many people in the community, this positions OpenTofu as a logical and straightforward path to return to business as usual.

Moreover, adopting OpenTofu also serves as a way to tap into the wellspring of community contribution that dwindled for Terraform following the licensing change.

As a vendor-neutral Linux Foundation project, OpenTofu remains true to Terraform’s original open-source ethos, drawing on the collective wisdom and diversity of the FOSS community.

This commitment to openness, reinforced by the project’s transparent governance through a multi-vendor technical steering committee, ensures that community innovation can flourish.

But don’t just take my word for it. Form your own opinion by joining our Slack community (2,100 members and growing), or follow the discussions on our GitHub repo. You can also keep up with our project updates and engage in open discussions by following us on Twitter or LinkedIn, and by joining our office hours.

Looking Forward

Switching to OpenTofu isn’t the right move for everyone. For example, if you’re deeply embedded in Terraform Cloud, it may be more practical to stick with HashiCorp.

However, the undeniable fact is that currently, the effort to experiment with OpenTofu is as low as it’ll ever be.

So, if you are looking for a true open-source project backed by a vibrant and enthusiastic community, or you just foresee a future where OpenTofu would fit into your tech stack, the best time to dip your toes in the water is right now.
May 8, 2024
The Problems With Golden Templates That No One Wants to Talk About

Without question, golden templates (or golden images) simplify the process of deploying resources in on-premises and cloud infrastructures. These templates can save time. They ensure consistency by setting up a computing environment that matches the specifications. The templates also encourage component reuse, rather than requiring individual users to manually compose infrastructure as code (IaC) from scratch for each application. For common types of projects, the infrastructure is frequently the same, so why force busy developers to rewrite it for everything they want to deploy?

Golden templates were a great first step in making IaC more accessible for end users, including people who might otherwise find it difficult to compose IaC. Preset standards can plug-and-play with the users’ applications and projects.

That’s far better than the alternatives. Nobody has to ask those users to learn new tools, such as Terraform, AWS CloudFormation, or Ansible, each of which has its own syntax and nuances. Nor are those end users expected to understand the infrastructure, such as servers, storage, networks and operating systems. It’s akin to a PowerPoint template that is pre-equipped with a brand’s colors, fonts, icons, and predefined layouts.

Golden templates make it easier to get started with IaC and deliver something that looks right, with all the right pieces.

Or, at least, that was the goal. Despite the strengths of golden templates, there are still issues to resolve.

The Problems Golden Templates Address Well

Making IaC repeatable makes sense. It builds in infrastructure security, consistency, and performance by removing the likelihood of manual errors, ensuring best practices are followed, and enforcing the use of approved modules, providers, and resources.

Golden templates work fairly well for simple things you need to do all the time, such as deployments where the infrastructure is the same every time and the underlying codebase rarely changes. There’s no need to reinvent the wheel for every application deployment, so it makes sense to use a template that describes which modules to use and in which order. As with that PowerPoint template, the approach is relatively simple and it doesn’t require a platform engineer’s time and attention.

Saving time and improving efficiency are clear wins in themselves, and golden templates theoretically offer both of those, alongside security. For common use cases, templates can make it easy for end users to deploy their work without sending tons of requests to the platform team or exposing their organization to unnecessary risk.

Ultimately, though, like any template, golden templates are fragile.

Where Golden Templates Fail, and Why

Golden templates work great on Day One, but two things happen fairly quickly that throw a wrench into the works: updates to components and updates to the templates themselves.

Templates often include versioned components, such as modules and dependencies on different (often open source) projects.

What happens when you want to use the latest version of a module in your template? You have two options: update the whole template; or try to design your templates to always use the latest version, which has its own risks and pitfalls. Most companies want to test the latest versions before rolling them to production. And you don’t want to update your template one week only to do it again the next week when another component gets a new version.

That brings us to the process of rolling out new versions of the template, whether that happens frequently or once a year. Updating golden templates means they need to be in a version control system or some other distribution channel that your development teams actively use. If you’re lucky, you’ve built those templates into your build/deployment pipelines.

But what happens when not every team is cleared for updates? You want to make sure that your developers use the version that captures all the latest rules, best practices, and institutional knowledge from your platform engineering team. You also need to ensure that you’re not disrupting their existing infrastructure. And, if you’re not building templates into pipelines, you had better keep your fingers crossed that your end users are choosing the updated version of the template appropriately (rather than grabbing an outdated template).

That makes the process more complicated, when the entire point was to simplify. You now need to keep track of who uses which version of the template. You also need to track versions of the various components you need to support, maintain, or plan to manually upgrade eventually. And, as with any other template, if you mess up the golden template everyone uses, the problems trickle down. Everything using that template includes those same problems baked in, creating a sprawling remediation nightmare.

The Service Catalog Alternative… Maybe

Of course, a single template can’t fit all deployment needs, so many companies create service catalogs of templates. These catalogs allow developers to make template selections through multiple-choice questions or by filling in the blanks. This solves the challenges of relying on a single template for many use cases.

However, the service catalog model often requires end users to select infrastructure correctly for their use case, which in turn requires that you wrote the questions or directions expertly to guide them without requiring them to have additional infrastructure knowledge. As anyone who has tried to write instructions or guides for anyone quickly learns: That’s hard to do well! Everyone thinks about things differently, and a straightforward direction for me might be really confusing for someone else. Templates and service catalogs don’t escape this issue.

All of this assumes that every template is equal and that every developer who uses a template has the same baseline knowledge. When you share a template with developers who know about infrastructure, they can fill out their part quite easily. For developers who don’t understand infrastructure, however, it’s much harder to fill in the blanks. They may need to turn to platform engineering for help (or make inaccurate guesses, which is arguably worse), and that undercuts the benefits of the template.

Golden templates never solved the problem of changing application needs, either. Templates are, by design, one size fits all. What happens when an application developer uses a template that just barely fits the needs of their application to get it into production, and then updates that application to have even higher infrastructure requirements? Is that developer going to assess whether the updated application has new requirements? Or are they just going to keep using the same template? The developer may not pull down a new golden template version, consider whether they need an extra database, how expanded capacity may change requirements, or how the scale the application is running at has changed – and the developer is less likely to do so when they are not connected to the day-to-day maintenance of the application’s infrastructure. The result is an application that isn’t set up to handle the load and therefore fails.

The reverse is just as likely. It’s too easy to deploy applications that are over-provisioned, and therefore taking up valuable space and resources. That also creates a wider surface for potential attacks.

Often, developers need to revisit an application’s infrastructure requirements, and templates don’t require or encourage that thought process.

How to Ensure Secure, Efficient IaC

We do not want to give up the positives of golden templates: automation of repetitive tasks, creating knowledgeable guidelines for users with less infrastructure knowledge, and more ability to build in known best practices and secure patterns.

But if templates are too fragile to use to reliably deliver secure and efficient IaC, how can organizations solve these problems while still gaining the benefits? Shift your thinking from a template where developers fill in the blanks, to asking how you and your organization can generate IaC from standards.

Standards are guidelines your organization defines for how to build and structure infrastructure. Standards establish the requirements to ensure your applications and projects are deployed such that they meet the needs of your business, the industry, and your customers.

Standards change significantly less often than templates, which implicitly combine standards with module and other component versions. Instead of placing the burden on developers to remember and align to these standards, apply your policies and standards automatically when IaC is generated, eliminating the need to scan and fix it after creation. This approach enables developers to deploy applications faster and platform engineers to provision infrastructure more easily because there’s no need to worry about whether the IaC is secure and efficient.

Photo credit: Neeqolah Creative Works on Unsplash

May 1, 2024
IBM Confirms: It’s Buying HashiCorp

In the hybrid-cloud world, we all knew that HashiCorp, makers of the popular infrastructure as code (IaC) tool Terraform, was looking to be acquired.

We also knew that switching Terraform’s open source Mozilla license for the Business Source License (BSL 1.1) had left plenty of unhappy Terraform developers and partners in its wake. Those open source users created a Terraform fork called OpenTofu. OpenTofu took off quickly, causing lawyers to get involved.

But that kerfuffle didn’t stop IBM from announcing it was buying HashiCorp for a cool $35 per share, a deal adding up to a premium value of $6.4 billion.

IBM chairman and CEO Arvind Krishna announced the acquisition in a Q1 financial call. Krishna remarked, “As generative AI continues to evolve, the complexities in tech strategies have significantly increased. HashiCorp’s stellar record in simplifying this modern infrastructure sprawl makes them an ideal addition to IBM’s hybrid cloud vision.”

The HashiCorp acquisition fits neatly with IBM’s 2019 acquisition of Linux and cloud power Red Hat. From a technical viewpoint, Ansible and Terraform are well-suited to work together. IBM clearly hopes that HashiCorp Terraform and Vault, its secret and encryption manager, will help strengthen its hybrid cloud and DevOps efforts.

HashiCorp’s co-founder and CTO Armon Dadgar is enthusiastic about joining forces with IBM. “This merger is not just a growth trajectory for HashiCorp but a gateway to bring advanced cloud solutions to a broader audience, accelerating innovation and enterprise efficiency in cloud management.”

HashiCorp’s CEO, Dave McJannet, agreed. “IBM’s leadership and rich history in hybrid cloud innovations make it the perfect home for HashiCorp as we step into a vigorous phase of our growth.” IBM sources say that McJannet will report to Rob Thomas, IBM’s senior VP in charge of software.

The Industry Responds

“According to our research, more than six tools are used by organizations to support cloud computing and automation. The combination of IBM, Red Hat, and potentially HashiCorp product portfolios suggests changes in cloud computing and the overall automation landscape,” said Paul Nashawaty, a practice lead at The Futurum Group. “It is an alliance of seasoned professionals that goes beyond a simple technology merger to give organizations the tools and insights they need to manage the complex web of modern IT with efficiency and foresight.”

While HashiCorp’s licensing move seems to have paid out for HashiCorp’s investors, IBM’s investors don’t seem so keen on the deal. In early after-market trading, IBM’s shares declined. Even before the BSL 1.1 changeover appeared, HashiCorp’s revenue growth had been slowing.

It should be noted that while IBM’s earnings were lackluster in most areas, the financial call called out positive news in software. Krishna and CFO Jim Kavanaugh credited Red Hat, especially its OpenShift hybrid cloud platform and its Ansible DevOps platform, for this growth. Kavanaugh said, “Both offerings also continued to take market share this quarter.” Specifically, Red Hat’s revenues grew 11% year-over-year, with OpenShift growing over 40%.

Setting the business aspects aside for now, it’s an open question what this will mean for Terraform and OpenTofu. Will IBM return Terraform to its open-source roots? Will IBM, a top Linux Foundation member, try to unite the warring sides? Stay tuned. It’s going to be a bumpy ride.

Photo credit: Hamid Roshaan on Unsplash.

April 24, 2024
OpenTofu Denies Hashicorp’s Code-Stealing Accusations

It’s been ages since copyright conflicts have been an issue in open source circles, but they emerged when HashiCorp accused OpenTofu of “not respecting the terms of its BSL license governing its Terraform codebase.” OpenTofu denied Hashicorp’s accusation that it had stolen any of the Terraform Business Source License (BSL) code. Now, OpenTofu‘s law firm, Gesmer Updegrove, has formally denied Hashicorp‘s accusation.

Hashicorp did more than claim OpenTofu had misappropriated some of its code. Its law firm, Wilson Sonsini, issued a cease-and-desist order against OpenTofu. This read in part, “This is a cease and desist demand to the supporters of the OpenTofu project. Specifically, OpenTofu has repeatedly taken code HashiCorp provided only under the Business Software License (BSL) and used it in a manner that violates those license terms and HashiCorp’s intellectual property rights. In at least some instances, OpenTofu has incorrectly re-labeled HashiCorp’s code to make it appear as if it was made available by HashiCorp originally under a different license.”

The order continued, “Therefore, we demand that OpenTofu … cease and desist from further violations of HashiCorp’s BSL license and infringement of HashiCorp’s copyrights. If OpenTofu does not comply, we reserve all rights, including the right to send DMCA takedown notices to Github or any other third-party hosting or source code repository provider, and the right to initiate litigation to stop further violations.”

OpenTofu replied, “The OpenTofu team vehemently disagrees with any suggestion that it misappropriated, mis-sourced, or otherwise misused HashiCorp’s BSL code. All such statements have zero basis in facts.” In addition, it said, HashiCorp’s claims of copyright infringement are completely unsubstantiated.

Where’d the Code Come From?

As for the code in question, OpenTofu claims it can clearly be shown to have been copied from older code under the Mozilla Public License (MPL) 2.0. “HashiCorp seems to have copied the same code itself when they implemented their version of this feature. All of this is easily visible in our detailed SCO analysis, as well as their own comments.”

In a detailed source code origination (SCO) examination of the problematic source code, OpenTofu stated that HashiCorp was mistaken. “We believe that this is just a case of a misunderstanding where the code came from.” OpenTofu maintains the code was originally licensed under the MPL, not the BSL. If so, then OpenTofu was perfectly within its right to use the code in its codebase.

OpenTofu’s attorney said, “Both the OpenTofu files to which you refer and HashiCorp’s Terraform files to which you compare them are both derived (at least to some degree) from the pre-fork MPL-2.0 files – code that was made publicly available under the MPL.” Therefore, “To my client’s knowledge, none of the Terraform code subject to the BUSL has been improperly copied, incorrectly sourced, or used for any purpose.”

The attorney went on to say that HashiCorp should have known this. “[W]hile you complain that a comparison of HashiCorp Terraform code to OpenTofu files show ‘substantial similarity’ between the two, the diff files you attach to your Letter actually show the compared files to be quite dissimilar.”

The OpenTofu lawyer continued, “Going forward, we would be open to establishing developer liaison contacts between OpenTofu and Terraform, who would be available to review and address any intellectual property concerns with submitted contributions to OpenTofu or Terraform.”

Laferra concluded, “In the future, if you should have any concerns or questions about how source code in OpenTofu is developed, we would ask that you contact us first. Immediately issuing DMCA takedown notices and igniting salacious negative press articles is not the most helpful path to resolving concerns like this.”

So, what’s next? Having covered more than my fair share of intellectual property legal conflicts, I don’t expect HashiCorp to back down. I fear that this issue will be resolved in the courts, not in Git repositories.

Photo credit: Maarten van den Heuvel on Unsplash

April 11, 2024
The IaC Weight on DevOps’ Shoulders
It’s often challenging for established tech companies to adopt modern DevOps practices around infrastructure-as-code (IaC). It’s not so much in adopting the syntax, but rather the DevOps culture that IaC is part of for engineering organizations.

Organizations often struggle to transition from traditional operations models to collaborative DevOps culture, which is a byproduct of transitioning infrastructure operations to code.

In many cases, legacy tendencies persist, while IaC duties stay siloed within ops-centric DevOps teams—which is the exact opposite of the benefits of leveraging powerful technology like infrastructure-as-code. What this ultimately creates is bottlenecks in development velocity and product iteration.

With all this in mind, it begs the question: What changes can enable a smoother IaC journey?

The Pitfalls of IaC Silos

Having DevOps teams shoulder the entire IaC burden is the exact anti-pattern we’re looking to avoid when truly transitioning to DevOps culture in our organizations. What many organizations start to experience is these familiar pain points that start to emerge:

1.) Knowledge Gaps Widen

Developers—by nature of their day job as software engineers—often lack access, visibility and context into infrastructure changes. Yet, with more and more infrastructure requirements shifting left, they are left (pun intended) with insufficient capabilities to manage these growing needs.

This is compounded when they are faced with minimal cloud permissions. Limited permissions make it difficult to experiment and familiarize themselves with cloud tools and capabilities the way they would with code and its parts of the stack.

Meanwhile, DevOps teams—as the exact mirror image and the nature of their role in the organization—don’t grasp application needs and objectives. This often results in a widening chasm of broken communication and collaboration around IaC initiatives.

2.) Collaboration Suffers

A common tool that exacerbates the problem is a ticketing and support platform. Developers, as their modus operandi, lacking a shared understanding of the domain, will often resort to opening tickets to achieve their goals. This creates a culture of inefficient cross-team communication via ticketing systems.

This doesn’t mean the ticketing platforms are the problem, it just results in inefficient communication methods, as developers don’t always know what questions to ask.

What this does mean is that DevOps engineers operate largely on instinct, being mostly in the dark on the real application requirements that can’t all be communicated in a simple Jira ticket. This results in final solutions that rarely satisfy needs on the first attempt and take longer to deliver.

3.) Releases Slow to a Crawl

DevOps teams hold the responsibility for conducting and orchestrating release engineering, on top of provisioning environments, maintenance, monitoring and other tasks that fall in the DevOps engineering domain.

This is why DevOps tasks can become a bottleneck, as they not only dictate deployment timing (due to important considerations of uptime and client disruption, among others) but are also tasked with juggling infrastructure changes across many and multiple environments.

This causes unpredictable delays in provisioning dev environments and other developer-oriented tasks that frustrate developers and stall production updates, and delivery velocity.

The Obstacles of Change

Recognizing these inefficiencies and shifting these responsibilities left became the popular way for DevOps leaders to distribute IaC duties. This allowed developers to gain sufficient autonomy to choose the solutions for their needs, while DevOps teams provided guidance and oversight for how to practically achieve these solutions.

However, like all things shift left, this comes with its own set of unique challenges. Adopting the DevOps paradigm faces fundamental obstacles that add further complications and complexity to these cultural transitions, including:
- Uncontrolled sprawl
- Ballooning costs
- Security concerns
On the one hand, distributing the IaC load lessens the burden on the DevOps teams, but the downside is that it becomes difficult to understand which resources are actually in use and which have been temporarily created for testing purposes.

With many owners creating resources on demand, once they are no longer needed, these leftovers create confusion around dependencies and make cloud platforms disorganized and difficult to maintain.

Just like enabling more hands to touch IaC creates greater sprawl and disorder, more users with less governance invite careless sprawl in terms of costs as well. This often results in duplicate and unused resources accumulating, wasting budgets that are currently tight, and every penny counts. With a lack of automation and oversight, environments grow messy and expensive.

The sprawl issues can also impact security, as expanding permissions raises valid security concerns that are intensified when clouds become disorganized and difficult to maintain.

Well-intentioned developers may misconfigure resources or expose sensitive systems, and without proper methods to manage drift or misconfiguration, this can pose real risks to organizations and systems. Another important aspect that also increases with less oversight is intentional insider risk.

From Silos to Shared Responsibility

OK, that must have been really disheartening—and you’re probably thinking you should run for the hills before attempting any transitions to shared responsibility.

But wait—don’t panic yet. It is possible for teams to redistribute IaC responsibility while avoiding these pitfalls.

Below are some key factors to help you make the transition smoother in the short and the long term:
- Automate everything—If DevOps practices have taught us anything, it is that manual cloud environment maintenance doesn’t scale. Automation has been the backbone of much of the evolution we have undergone when it comes to infrastructure operations, and this has been powered by the programmability and everything-as-code mindset—from IaC to policy-as-code (PaC). This automation-based optimization makes it possible to reduce neglected resources through regular and automated analysis and response. What you get is cost savings and the prevention of runaway sprawl.
- Empower through governance—Remember when we spoke about automation and governance? Well, programmatic policy enforcement is a good way to constrain unwanted resource proliferation without manual intervention. Governance comes in to handle auto-remediation of misconfigurations or other anti-patterns that do not align with automated policies, which simplifies environment management at scale. Importantly, governance, if done right, can be a tool to empower teams, break silos and encourage autonomy. Having the right guardrails in place is how you instill confidence and provide helpful direction.
- RBAC for the win—Role-based access control (RBAC) limits permissions to appropriate needs, establishing important guardrails that can help prevent some of the issues we spoke about above, including sprawl and security concerns.
RBAC will enable developers to gain the necessary freedom while minimizing security risk. Minimum required access encourages responsibility and constraint.

Now that you have a solid grasp of the principles, it’s time to put them into action!

Fortunately, there are numerous solutions at your disposal. You can jump in and build them yourself using capable tools like Jenkins or Atlantis.

Alternatively, for a more comprehensive approach, there are excellent commercial IaC management tools on the market. These can assist you in reaching your objectives swiftly and with greater efficiency.

An Evolutionary Vision for IaC and DevOps

As technology complexity increases, no single team can handle infrastructure alone. Dev and Ops must unite abilities to shepherd modern systems and the underlying code powering them. This requires closing knowledge gaps through collaboration and communication

With a thoughtful process, IaC automation, governance guardrails and the right access policies, tech leaders can guide this cultural transition smoothly without exposing the organization to unnecessary risk and spending.

When done right, Shared IaC responsibility will boost velocity, agility, and autonomy—the trifecta of crucial traits that quantify and qualify elite engineering teams and differentiate companies in a competitive landscape.

Interested in learning more about env0 to manage your IaC at scale and confidently?

Stop by Booth L12 at KubeCon EU to speak with an expert or see a demo!

To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.
March 13, 2024
Platform Engineering: The 2024 Game-Changer in Tech

Platform engineering, a key driver in the ever-evolving technological landscape, is poised for significant momentum in 2024. This article explores the reasons behind its rising popularity, the types of organizations that stand to benefit, potential drawbacks and the crucial role of test data management (TDM) in this context.

What’s Driving Demand for Platform Engineering?

Platform engineering is anticipated to gain significant momentum in 2024 as it offers crucial advantages in accelerating business value, reducing cognitive loads and enhancing the efficiency of application development and management processes. According to Gartner, by 2026, about 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery. This trend underscores the increasing adoption and importance of platform engineering in the tech world.

The growing popularity of platform engineering can be attributed to several factors:

• Developer productivity: Enhancing developer productivity is a key driver, accounting for approximately 21% of the reasons for its popularity.
• CI/CD pipeline implementation: About 20% of the popularity is driven by the need for efficient continuous integration and continuous delivery pipelines.
• Standardization of tools and processes: Another 20% comes from the necessity to standardize tools and processes.
• Security enhancements: Security improvements contribute 20% to its popularity.
• Infrastructure-as-code (IaC): Finally, 19% is driven by the adoption of infrastructure-as-code (IaC) methodologies.

In addition, platform engineering addresses the inefficiencies caused by the decentralization of tools and processes in DevOps, especially as organizations scale. This approach provides a more organized and centralized framework, which is essential in managing the complexities of modern software development environments.

As highlighted by Gartner, platform engineering’s potential to offer a frictionless self-service experience to customers and business partners is a significant factor in its rising popularity. This adaptability is crucial in an era where technology and market demands are rapidly evolving, requiring businesses to remain agile and competitive.

These factors collectively make platform engineering a strategic approach for software engineering organizations, particularly looking forward to the year 2024 and beyond.

Key Benefits of Platform Engineering

While it isn’t a fit for every organization, there are some shared attributes of companies that are most likely to benefit from it.

If your organization is seeking to:

• Reduce tool sprawl by standardizing tools and services
• Improve productivity through automation, leading to faster development cycles
• Establish governance frameworks for software development
• Increase scalability through technologies like containerization and orchestration
• Enhance security by integrating automated security scanning tools into the CI/CD pipeline,

then platform engineering may be right for you. These benefits are especially relevant for businesses facing complex software development cycles and those in need of robust, scalable, and secure solutions.

Drawbacks of Platform Engineering

However, platform engineering is not without its challenges. Increased complexity in infrastructure can be daunting for small-scale teams. High dependency on the platform may lead to difficulties in switching technologies or providers. Additionally, compatibility issues with existing development tools and technologies can slow down processes.

The Crucial Role of Test Data Management (TDM)

In a platform engineering environment, test data management (TDM) becomes a pivotal component. TDM’s importance lies in its role in agile development, ensuring test accuracy and reliability, thus enhancing the effectiveness of the testing process. Agile methodologies, which divide projects into smaller sprints, rely heavily on TDM for providing high-quality, compliant test data in a continuous integration and deployment model.

TDM addresses the challenge of managing data fragmented across various applications, environments and compliance levels. An entity-based TDM approach enables testers to provision test data on demand, streamlining the testing process and reducing the risk of data breaches. Ultimately, TDM practices significantly reduce time, cost, and risk in software delivery, making it a vital component in modern platform engineering strategies.

As technology continues to evolve rapidly, the role of platform engineering, bolstered by effective TDM, becomes increasingly crucial. Companies that do not adapt to these changes by integrating components like TDM into their platform engineering strategies will risk falling behind in the competitive tech market. Therefore, embracing these innovations is essential for survival and success in today’s fast-paced technological landscape.

March 5, 2024
Survey: Significant Shift in IaC Tool Preferences

A survey of 350 IT professionals conducted by Firefly, a provider of a platform for managing cloud computing environments, suggests usage of infrastructure-as-code (IaC) tools is evolving rapidly in the cloud computing era.

For example, more than 40% of the respondents reported that, to some degree, they are already using OpenTofu, an open source IaC tool that was launched as a fork of the Terraform IaC in the wake of changes to the software licensing terms it is made available under by HashiCorp. Well over half said they planned to use OpenTofu in the future. That compares to 60% using Terraform today, but only just over 20% said they planned to use Terraform in the future.

The licensing changes HashiCorp implemented only affect third-party entities within a third-party platform. However, the survey indicates the number of end users affected by that change is more substantial than anticipated. A full 56% of respondents described the changes made to the Terraform license as disruptive.

The survey also finds there is a lot more diversity in terms of the types of IaC tools being employed, with both the IaC tools from Pulumi and the open source Crossplane framework being advanced under the auspices of the Cloud Native Computing Foundation (CNCF) also gaining traction. The report finds more than 40% of respondents are also using Pulumi, with well more than half planning to use that IaC tool in the future.

Crossplane, meanwhile, is now being used by 40% of respondents, with 60% planning to adopt it.

Conversely, the report also notes the number of organizations planning to migrate away from CloudFormation from Amazon Web Services (AWS) is rising. While well over 60% use AWS CloudFormation today, only a third plan to use it in the future.

Cindy Blake, vice president of marketing for Firefly, said as organizations employ additional cloud platforms, IT teams are clearly looking for tools that make it easier to centralize the management of multiple computing platforms. That trend will only accelerate as more DevOps teams adopt platform engineering as a methodology for managing the building and deployment of applications at scale, she added.

In fact, noted Blake, it’s already fairly common for IT teams to use two or more frameworks. The report finds 57% of respondents work for organizations that are using two or more frameworks. Overall, the report finds that more than 64% of respondents have also codified more than half of their cloud assets.

The number of organizations employing IaC tools to programmatically manage infrastructure is only going to increase, said Blake. The report, for example, found that three-quarters of respondents either already are or plan to use IaC tools to configure software-as-a-service (SaaS) application environments. In the longer term, advances in artificial intelligence (AI) will make it simpler for organizations to achieve that goal. Open source tools such as AiAC, for example, already make it possible to leverage large language models (LLMs) to generate code. Those types of tools will eventually eliminate the need to master specific programming languages to provision IT infrastructure as code, noted Blake.

It’s not clear how IaC tools within the context of DevOps workflow will evolve, but the one thing that is certain is that the number of types of tools being adopted has never been more diverse.

February 29, 2024