devopsy.kinsta.cloud

Tag: Terraform

HashiCorp Updates Terraform; Wider Cloud Infrastructure Developer Toolsets
As HashiCorp quietly slips into its new existence under the parent umbrella of IBM after this year’s $6.4 billion acquisition, the company held (one last?) HashiConf in Boston this week to set out its stall. That market stall is of course one populated by infrastructure lifecycle management and security lifecycle management products for hybrid multi-cloud environments.

Key among the announcements made by the company right now are updates to its Terraform Infrastructure-as-Code (IaC) cloud provisioning technology and its HCP Waypoint tool, which is designed to create internal developer platforms with templates and add-ons to manage application dependencies.

HashiCorp says that Terraform’s capabilities for infrastructure provisioning enable cloud-native developer teams to take a cloud-agnostic highly interoperable approach to multi-cloud management. It operates with a single workflow (i.e. a singular software code process within which all planning, testing, deployment and monitoring can happen) so that lifecycle management can be promoted as an essential function.

Terraform Stacks

Given the wider trend for compartmentalized containerized abstracted pre-configured convenience across all software application development toolsets and platforms, it is perhaps not a surprise to see HashiCorp put as much automation convenience as possible into Terraform. This now comes in the form of Terraform stacks.

Terraform stacks is a configuration layer update intended to simplify cloud provisioning and cloud resource management at scale by controlling “cross-configuration dependencies” between Terraform modules. Given that software application dependencies are fundamental, cross-configuration dependencies at the infrastructure provisioning level can also include configuration item hierarchies, where dependencies are an association between two configuration elements that would be rendered to an invalid state if both were not present and working correctly.

Stacks is essentially a way of grouping different Terraform cloud infrastructure modules (we could also call them components) together in order to centralize maintenance and interdependency tasks. While this function has been around in private beta for a while, HashiCorp is now moving this technology to public beta. Terraform stacks are controlled from a Terraform configuration (aka Terraform-config), which the company describes as a complete document in the Terraform language that tells Terraform how to manage a given collection of infrastructure. A configuration can consist of multiple files and directories.

Multiple Infrastructures, Multiple Environments

“With Terraform Stacks, we’re reimagining infrastructure as code, making it easier to build and deploy the same infrastructure multiple times, across multiple environments, regions, landing zones, or accounts within a cloud provider,” said Armon Dadgar, CTO and Co-Founder, HashiCorp. “With new features across Vault and Boundary, we’re bringing important management capabilities to our security portfolio, including auto-rotation, dynamic secrets and secret scanning to provide full lifecycle management for security.”

Also central to the product announcements brought forward right now is HashiCorp Cloud Platform (HCP) Waypoint. This is an internal developer portal with templates to codify golden patterns (proven, reusable and validated software patterns for infrastructure and security automation workflows) for day zero provisioning, add-ons for day one (24 hours after deployment to live production status) and updates to manage Day N (the decommissioning point at which any software system reaches end-of-life) operations.

Other updates include:
- HCP Vault Secrets adds new lifecycle management features, including auto-rotation (generally available), dynamic secrets (public beta), and dynamic cloud credentials for HCP Terraform (public beta) to support secrets lifecycle management for enterprises.
- Terraform migrate (public beta) automates the migration of common DIY workflows from Terraform Community Edition to HCP Terraform or Terraform Enterprise.
- Nomad GPU enhancements in 1.9 (generally available) adds advanced GPU scheduling to support demanding AI workloads. This includes support for multi-tenant workloads sharing GPUs to maximize utilization and resource quotas to efficiently broker access to shared resources.
Cloud Management, Management

What’s happening here is logical, inevitable, redolent of wider automation trends and (arguably) quite welcome. As cloud lifecycle management technologies now evolve at the infrastructure-first level, the gloomy specter of spiralling hyperscaler costs will naturally make firms more interested in operational streamlining technologies of this kind.

With distributed diverse divergent application types also coming to the fore (edge computing is getting so much smarter as we know – and that’s not the only zone where compute and analytics are in a vortex), organizations will want lifecycle management tools that enable them to be compliant with governance legislature and, in many cases, just to keep their heads above water.

It is, in other words, cloud management management.

Whether HashiCorp remains as independently branded and operated as Red Hat under IBM is more open to question. Only Big Blue knows.
October 18, 2024
To Monetize HashiCorp, IBM Must Perform an Unprecedented Balancing Act

IBM’s multi-billion dollar deal to buy HashiCorp has left the hybrid cloud industry guessing what the future holds in store for the most popular provider of tools for DevOps and infrastructure-as-code (IaC).

HashiCorp was rumored to have been looking for an acquirer for several months before the deal was announced. Despite enjoying top dog status in the IaC space thanks to its flagship Terraform app, the company had been struggling to translate its popularity into profitability, resulting in immense pressure from shareholders to do something drastic.

HashiCorp’s financial struggles were all too apparent when it published its fiscal 2023 financial results, revealing a $274 million loss on a total revenue of $583.1 million that year.

Its inability to drive a profit is thought to be what prompted the company to make the controversial decision of replacing Terraform’s open-source Mozilla license with a more restrictive Business Source License (BSL 1.1), which caused an uproar among developers and partners. Such was the Terraform community’s dismay that it quickly responded by creating a Terraform fork, eventually called OpenTofu, which rapidly accumulated a big following.

IBM’s challenge is to turn HashiCorp’s fortunes around, and there has been a lot of speculation over how the company intends to do that. It can go one of two ways — either it can make Terraform more restrictive, binding it more tightly with its own cloud products and services, or it can perform a rapid U-turn and embrace the open-source community once more.

Open Source is a Tough Business

HashiCorp is not alone in its struggles. The challenges of monetizing open-source software are well documented, and many of HashiCorp’s peers have opted to take a similar approach, making their products more restrictive in an effort to stop bleeding cash. Redis, for instance, recently mirrored HashiCorp’s move when the company announced it was dropping its Berkeley Software Distribution license in favor of a new Redis Source Available License and Server Side Public License model, placing restrictions on how its database software can be modified. Elasticsearch made a similar move in 2021.

The problem with open-source software is that its development is anything but free. While there are a handful of projects led by hobbyist developers, the reality is that most are under the control of big technology firms that employ armies of developers to maintain them. Google, for example, is the main contributor to the Kubernetes project, and it pays hundreds of developers to work on that initiative full-time. This is possible because the company makes money from selling cloud infrastructure services that support Kubernetes workloads.

However, other companies cannot do this, and HashiCorp is a prime example. For most users, the open-source version of Terraform had already provided everything DevOps teams needed to orchestrate their cloud environments, and they saw little reason to pay for the premium Terraform Cloud product, even if it was a little easier and had some nice add-ons.

HashiCorp said in October 2023 that it was hoping to become profitable by the second half of its fiscal 2025 year, but the growth metrics that the company has reported in recent quarters make this goal look like a longshot at best.

Open or Closed? That is the Question

One of the reasons why Terraform quickly became the most popular IaC offering was its cloud-agnostic nature. Most organizations these days are multi-cloud and Terraform gives them a common configuration language.

For IBM, maintaining this cloud-agnostic nature of Terraform might be perceived as a conflict of interest. With IBM Cloud having a tiny market share of just 1.8%, it seems to make little sense to keep supporting a tool that enables people to use its competitors.

This has led to concerns that IBM plans to bind Terraform more tightly with its own cloud offerings. The company may opt to prioritize integration with IBM Cloud at the expense of other cloud infrastructure platforms. It also seems likely that IBM will want to integrate Terraform with Red Hat, which is now its biggest revenue growth driver. Interoperability could be limited to IBM’s products, and access to its source code could be limited to paying users only, to maximize revenue.

The Terraform community is unlikely to appreciate such a move, given its reaction to HashiCorp’s licensing changes last year. In an op-ed in Forbes, Futuriom analyst Scott Raynovich highlighted the extremely negative response by many Terraform users, accusing the company of turning its back on its open-source roots.

“The open-source licensing had enabled thousands of partner companies to incorporate Terraform and other HashiCorp coding into products that are sold commercially,” Raynovich noted. “The announcement also lit a fire under the age-old debate of what’s cool and what’s not cool in the open-source community. As you can imagine, a lot of people thought it wasn’t very cool.”

However, there is plenty of optimism that IBM could be good for HashiCorp. In particular, there was a positive reaction from the Terraform community on Reddit, with many members expressing their hope that IBM would make Terraform open source again. While a more restrictive approach might make sense from a revenue perspective, IBM will also be well aware of the benefits of such a move.

Sebastian Stadil, a co-maintainer of the forked OpenTofu project, recently told The New Stack he would be open to reconciling with Terraform if ever HashiCorp decided to return to a more open licensing model. He also revealed that OpenTofu’s maintainers have held confidential talks with IBM, without providing any details.

It is clear that OpenTofu, with its less restrictive model, represents a significant threat to Terraform; so IBM might well be motivated to eliminate one of its biggest potential challengers by returning to the open-source path.

Terraform is Not the Only Show in Town

A decision by IBM to continue along the path laid out by HashiCorp’s management is unlikely to result in everyone abandoning Terraform. For many users, it will be far too inconvenient or challenging to swap it out for some other kind of cloud provisioning tool, especially if subsequent versions of OpenTofu are less compatible with Terraform over time.

Although Terraform is widely seen as the leading IaC tool, it is not so popular that it can influence the entire industry. DevOps practitioners are exemplified by their resourcefulness and their innovativeness, and they will always be able to find an alternative way forward.

In a recent panel discussion hosted by IaC management framework company Env0, Ohad Maislish, CEO, commented on the tool ecosystem around Terraform and how that might translate for companies that embrace alternatives. “OpenTofu is a unique example of something that from day one already has a very mature ecosystem of adjacent solutions and technologies,” he said. Today, many cloud providers already offer open-source modules to support deployment via OpenTofu.

OpenTofu isn’t the only alternative, either. Pulumi is another option, though it also has financial sustainability concerns, with a business model that seems similar to HashiCorp’s old, pre-license switch. Then there is Crossplane, which has a more Kubernetes-centric flavor and is backed by the Cloud Native Computing Foundation. A third option is for Terraform users to instead go with the dedicated offerings of the major cloud infrastructure providers, such as Cloud Formation for AWS, Config Controller in Google Cloud and Bicep on Microsoft Azure.

Striking the Right Balance

Whichever path IBM decides to take, the tech giant’s leadership will do well to remember that in the DevOps industry, there is a mindset that emphasizes principles over the tools required to do the job.

Should Terraform evolve in a direction that displeases the DevOps community, the industry will simply switch to a new standard. After all, one of the things that characterizes the DevOps movement is the ability of teams to quickly adapt and embrace new tools and systems that allow them to get things done faster and more efficiently.

For now, IBM is keeping its plans for Terraform under wraps. The challenge it faces is to keep the majority of Terraform’s users happy, while finding a way to turn a profit. Getting it right means performing an extremely delicate balancing act that has few precedents. It may not be impossible, but it will be an impressive feat if IBM can do it.

July 29, 2024
Scaling with Terraform: How to Manage Infrastructure at Scale

Managing large-scale infrastructure is a complex task that requires a strategic approach, especially when using infrastructure-as-code (IaC) tools such as Terraform. As organizations expand, their infrastructure must scale accordingly; however, it often becomes more intricate and challenging to manage. Terraform provides a robust framework for defining, provisioning and managing infrastructure, but scaling these operations requires a deep understanding of both Terraform and best practices for large-scale infrastructure management.

In this article, we will delve into effective strategies and practices for managing Terraform infrastructure at scale.

Benefits of Using Terraform for Large-Scale Infrastructure

● Scalability: Terraform is designed to handle complex infrastructures with ease. Its modular architecture and state management capabilities allow you to scale your infrastructure efficiently.
● Consistency: By defining IaC, Terraform keeps your infrastructure consistent across different environments and reduces the risk of configuration drift and errors.
● Automation: Terraform is a great help when it comes to automating provisioning and management of infrastructure, enabling you to deploy resources quickly and reliably. This automation is crucial for large-scale environments where manual intervention is impractical.
● Collaboration: Terraform’s configuration files can be versioned and shared, promoting collaboration among teams. The collaborative approach ensures that everyone is on the same page and reduces the likelihood of conflicts.

Strategies for Managing Terraform at Scale

As you scale your infrastructure, the complexity of managing resources and ensuring stability increases exponentially. The strategies outlined below will help you scale effectively with Terraform.

Structuring Your Terraform Code

Modularization
A key strategy for managing Terraform at scale is modularization. Breaking down your Terraform configuration into reusable modules enhances readability, maintainability and reusability. Modules are self-contained packages of Terraform configurations that manage a specific set of resources. By utilizing modules, you can standardize configurations across different environments and projects, reducing redundancy and errors.

Organizing by Environment
Separate your configurations by environment (e.g., development, staging and production). This structure prevents accidental changes in production and allows for environment-specific customization.
Example Directory Structure:
├── environments
│ ├── dev
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ ├── provider.tf
│ │ ├── terraform.tf
│ │ ├── terraform.tfvars
│ │ └── variables.tf
│ └── prod
│ ├── main.tf
│ ├── outputs.tf
│ ├── provider.tf
│ ├── terraform.tf
│ ├── terraform.tfvars
│ └── variables.tf
└── modules
├── ec2
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
└── network
├── main.tf
├── outputs.tf
└── variables.tf

Terraform workspaces are used to manage multiple environments (e.g., development, staging and production) within the same configuration. The isolation ensures that changes in one environment do not impact others.

State Management

Proper state management is crucial when dealing with large-scale Terraform deployments. The Terraform state file contains mappings between your configuration and the deployed infrastructure. Using remote state storage, such as AWS S3 with DynamoDB for state locking, ensures consistency and prevents conflicts.

terraform {
backend “s3” {
bucket = “bucket-name”
key = “path/to/terraform.tfstate”
region = “us-east-1”
“encrypt” = true
dynamodb_table = “terraform-locks”
}
}

Optimizing Resource Configuration and Performance

Here are a few tips to achieve optimal performance in large-scale environments:

● Maintain Consistency in Naming Conventions: Follow a consistent naming convention for resources to make it easier to identify and manage them, especially in large deployments.
● Use Data Sources: Dynamically fetch information about existing resources to avoid hardcoding values.
● Minimize Resource Dependencies: Reduce dependencies between resources to enable parallel deployments and speed up provisioning.
● Apply Lifecycle Hooks: Control the order and conditions under which resources are created, updated or destroyed.
● Review and Update Regularly: Keep your Terraform configurations up to date with the latest features and practices.

Automated Workflows

Integrate with CI/CD Pipelines

Automate your Terraform workflows using CI/CD tools such as Jenkins, GitLab CI and GitHub Actions to minimize manual effort and ensure consistency across deployments.

CI/CD pipelines enable you to automate various stages of your Terraform workflows, including validation, planning and application of changes. This automation ensures that every change goes through a standardized process, reducing the risk of inconsistencies and errors. For example, when a developer submits a pull request with infrastructure changes, the pipeline can automatically trigger the validation and planning stages and notify the relevant team members for review. This process ensures that changes are peer-reviewed and meet the required standards before being merged and applied.

Advanced CI/CD Integration

Terraform Cloud, Terraform Enterprise or Spacelift offer additional features including remote runs, state management and policy as code. These services help manage Terraform at scale by providing a centralized platform for collaboration, governance and automation.

Policy Enforcement

As your infrastructure grows, enforcing policies becomes crucial to ensure compliance and security. Terraform provides Sentinel, a policy as code framework that allows you to define rules governing your infrastructure. By integrating Sentinel policies into your CI/CD pipelines, you can automatically enforce best practices and organizational standards. Alternatively, you can use Open Policy Agent (OPA) with Terraform to achieve these results.

Wrapping Up

Scaling infrastructure with Terraform requires a blend of strategic planning, best practices and the right tools. The key points above can help you effectively manage Terraform infrastructure at scale, ensuring your systems are resilient, scalable and compliant with organizational policies and industry standards.

July 8, 2024
Mind the Shift: Answering Common Questions About OpenTofu
I recently returned from KubeCon EU where I participated in the first-ever OpenTofu Day and an Evolution of Infrastructure-as-Code panel discussion. During the conference, dozens of engineers and tech execs asked me questions about the OpenTofu project. They wanted to ascertain if it made sense for them to switch to OpenTofu, whether for testing purposes or to make the IaC management tool a permanent fixture in their tech stack.

Many people shared similar questions and concerns which, notably, were still the same as those expressed during the GA release. Now, on the heels of the OpenTofu v1.7 release, I’m taking the time to summarize my answers to those frequently asked questions and to share my (understandably biased) insights on the advantages of adopting it.

What Does “Drop-In” Replacement Mean?

In our communications, we commonly refer to OpenTofu as a “drop-in replacement” for Terraform. Unsurprisingly, the most common question I was asked was, “What does ‘a drop-in replacement’ really mean?”

As a developer, I understand where such questions come from. “Drop-in replacement” sounds more like a marketing promise than a technical specification. With that in mind, let me clarify what we mean.

To try OpenTofu, follow these steps:
1. Install OpenTofu.
2. Use the tofu command to execute your previously terraform commands (e.g.,tofu plan instead of terraform plan, tofu apply instead of terraform apply, and so forth).
And that’s it.

You don’t need to change a single line of code in your Terraform configuration. You can continue using all your favorite providers and modules, public or private. They will all just work.

By “drop in” we mean literally the minimal possible amount of effort with the fewest possible adjustments.

What’s the Best Time to Make the Switch?

Another question I encountered, often implied rather than asked directly, was, “Should I start with OpenTofu now or wait a while?”

Setting aside my personal bias, I can sum up my answer with a simple, undeniable fact: No future scenario is likely to make the switch easier than it is right now.

Let me explain.

The ease of transitioning between OpenTofu and Terraform comes from their complete interoperability in 1.6 stable versions. The ability to switch between Terraform and its open-source successor with a simple command alias means there’s effectively no cost to trying OpenTofu.

To me, this suggests that if you’re considering giving OpenTofu a try—whether sooner or later—the best time to do so is now.

Will the Projects Diverge?

Inevitably they will. However, there’s more to consider.

Ease of migration is a top priority for us on the OpenTofu team, and it is equally important to our community. This commitment ensures that we will manage any divergence between versions carefully.

Take OpenTofu v1.7 as an example. This version introduces several features, such as the much-anticipated state file encryption. But importantly, all these additions are opt-in. This approach exemplifies our commitment to not locking users into OpenTofu against their preferences.

Starting with OpenTofu is essentially a low-effort trial. The worst-case scenario is that you end up with a free, open-source version of Terraform that includes some additional, optional features to explore.

This was the mindset of several people I spoke with at KubeCon who were adopting OpenTofu as a future-proofing strategy. Doing so would let them keep their options open and help them avoid being locked into any specific vendor-ecosystem or IaC management tool.

What About Community Support?

Support for the project is gaining momentum. Key players including Alpine, Brew, and GitLab have already moved away from BSL-licensed Terraform binaries to ensure long-term open-source interoperability and to address their unique needs.

These are just a few examples. The shift in HashiCorp’s licensing terms, compounded by its recent acquisition by IBM, has sparked concerns about long-term viability and compatibility. These are unlikely to go away anytime soon. For many people in the community, this positions OpenTofu as a logical and straightforward path to return to business as usual.

Moreover, adopting OpenTofu also serves as a way to tap into the wellspring of community contribution that dwindled for Terraform following the licensing change.

As a vendor-neutral Linux Foundation project, OpenTofu remains true to Terraform’s original open-source ethos, drawing on the collective wisdom and diversity of the FOSS community.

This commitment to openness, reinforced by the project’s transparent governance through a multi-vendor technical steering committee, ensures that community innovation can flourish.

But don’t just take my word for it. Form your own opinion by joining our Slack community (2,100 members and growing), or follow the discussions on our GitHub repo. You can also keep up with our project updates and engage in open discussions by following us on Twitter or LinkedIn, and by joining our office hours.

Looking Forward

Switching to OpenTofu isn’t the right move for everyone. For example, if you’re deeply embedded in Terraform Cloud, it may be more practical to stick with HashiCorp.

However, the undeniable fact is that currently, the effort to experiment with OpenTofu is as low as it’ll ever be.

So, if you are looking for a true open-source project backed by a vibrant and enthusiastic community, or you just foresee a future where OpenTofu would fit into your tech stack, the best time to dip your toes in the water is right now.
May 8, 2024
What OpenTofu 1.7 Means for DevSecOps

OK, hands up. Who deploys containers with Kubernetes, using infrastructure-as-code (IaC) to automate its provisioning, security, and maintenance? Right, many of you do. Until recently, that meant many of you also used HashiCorp’s Terraform. After HashiCorp changed its open source license, many Terraform developers forked the code into OpenTofu, and quite a few of you moved to OpenTofu.

If you’re among the DevOps who made that move, you have been rewarded with OpenTofu 1.7.0. The release’s major highlight is the introduction of end-to-end state encryption. This feature ensures that a state file remains shielded from unauthorized access across any storage backend.

While that capability matters to anyone doing DevOps, it notably works hand-in-glove with DevSecOps. With Terraform, you were strongly encouraged to use HashiCorp Vault as a centralized secrets manager. With this OpenTofu release, you can secure encryption passphrases through environment variables or opt for robust key management systems such as AWS Key Management Service (KMS), GCP KMS, or OpenBao, the open-source Vault fork.

OpenTofu lacks a policy-as-code enforcement framework, which HashiCorp Sentinel provides for Terraform. For OpenTofu, it would be best to use third-party, open source programs such as Open Policy Agent (OPA) that can embed security policies directly alongside an OpenTofu configuration. You can then store your OpenTofu and OPA configurations in a version control system like Git. This ensures that infrastructure changes are tracked and auditable and that they can be easily rolled back if necessary.

Another innovative addition in OpenTofu 1.7.0 is dynamic provider-defined functions. This allows providers to supply resources and offer native functions that can be used directly within OpenTofu code. An OpenTofu-exclusive feature lets these providers dynamically create custom functions based on user configurations, fostering a seamless integration of other programming languages. The OpenTofu team encourages users to explore this feature with the experimental Lua and Go providers.

Additional features are also included. Among them is the ability to mark specific resources for removal from the state file while retaining the underlying infrastructure. The new OpenTofu version also introduces loopable import blocks to simplify the bulk import of resources, greatly aiding large-scale migrations.

Building on its legacy as a drop-in replacement for Terraform 1.5, OpenTofu 1.7.0 promises easy migration paths, offers detailed migration guides, and includes comprehensive documentation with examples. That suggests an easy upgrade if you’re using Terraform for DevSecOps.

Since its inception, the OpenTofu community has seen remarkable growth. Despite eschewing user tracking, the registry’s usage has surged to over a million requests daily, with a doubling in just the past month. The release has drawn 65 unique contributors and garnered 20,000 stars on GitHub, underscoring a vibrant community engagement with over 200 new issues and numerous pull requests since January.

In short, OpenTofu appears to be a solid choice for both IaC and DevSecOps deployments, regardless of what happens with Hashicorp, IBM and Terraform.

Photo credit:Diana Polekhina on Unsplash

May 6, 2024
IBM Confirms: It’s Buying HashiCorp

In the hybrid-cloud world, we all knew that HashiCorp, makers of the popular infrastructure as code (IaC) tool Terraform, was looking to be acquired.

We also knew that switching Terraform’s open source Mozilla license for the Business Source License (BSL 1.1) had left plenty of unhappy Terraform developers and partners in its wake. Those open source users created a Terraform fork called OpenTofu. OpenTofu took off quickly, causing lawyers to get involved.

But that kerfuffle didn’t stop IBM from announcing it was buying HashiCorp for a cool $35 per share, a deal adding up to a premium value of $6.4 billion.

IBM chairman and CEO Arvind Krishna announced the acquisition in a Q1 financial call. Krishna remarked, “As generative AI continues to evolve, the complexities in tech strategies have significantly increased. HashiCorp’s stellar record in simplifying this modern infrastructure sprawl makes them an ideal addition to IBM’s hybrid cloud vision.”

The HashiCorp acquisition fits neatly with IBM’s 2019 acquisition of Linux and cloud power Red Hat. From a technical viewpoint, Ansible and Terraform are well-suited to work together. IBM clearly hopes that HashiCorp Terraform and Vault, its secret and encryption manager, will help strengthen its hybrid cloud and DevOps efforts.

HashiCorp’s co-founder and CTO Armon Dadgar is enthusiastic about joining forces with IBM. “This merger is not just a growth trajectory for HashiCorp but a gateway to bring advanced cloud solutions to a broader audience, accelerating innovation and enterprise efficiency in cloud management.”

HashiCorp’s CEO, Dave McJannet, agreed. “IBM’s leadership and rich history in hybrid cloud innovations make it the perfect home for HashiCorp as we step into a vigorous phase of our growth.” IBM sources say that McJannet will report to Rob Thomas, IBM’s senior VP in charge of software.

The Industry Responds

“According to our research, more than six tools are used by organizations to support cloud computing and automation. The combination of IBM, Red Hat, and potentially HashiCorp product portfolios suggests changes in cloud computing and the overall automation landscape,” said Paul Nashawaty, a practice lead at The Futurum Group. “It is an alliance of seasoned professionals that goes beyond a simple technology merger to give organizations the tools and insights they need to manage the complex web of modern IT with efficiency and foresight.”

While HashiCorp’s licensing move seems to have paid out for HashiCorp’s investors, IBM’s investors don’t seem so keen on the deal. In early after-market trading, IBM’s shares declined. Even before the BSL 1.1 changeover appeared, HashiCorp’s revenue growth had been slowing.

It should be noted that while IBM’s earnings were lackluster in most areas, the financial call called out positive news in software. Krishna and CFO Jim Kavanaugh credited Red Hat, especially its OpenShift hybrid cloud platform and its Ansible DevOps platform, for this growth. Kavanaugh said, “Both offerings also continued to take market share this quarter.” Specifically, Red Hat’s revenues grew 11% year-over-year, with OpenShift growing over 40%.

Setting the business aspects aside for now, it’s an open question what this will mean for Terraform and OpenTofu. Will IBM return Terraform to its open-source roots? Will IBM, a top Linux Foundation member, try to unite the warring sides? Stay tuned. It’s going to be a bumpy ride.

Photo credit: Hamid Roshaan on Unsplash.

April 24, 2024
OpenTofu Denies Hashicorp’s Code-Stealing Accusations

It’s been ages since copyright conflicts have been an issue in open source circles, but they emerged when HashiCorp accused OpenTofu of “not respecting the terms of its BSL license governing its Terraform codebase.” OpenTofu denied Hashicorp’s accusation that it had stolen any of the Terraform Business Source License (BSL) code. Now, OpenTofu‘s law firm, Gesmer Updegrove, has formally denied Hashicorp‘s accusation.

Hashicorp did more than claim OpenTofu had misappropriated some of its code. Its law firm, Wilson Sonsini, issued a cease-and-desist order against OpenTofu. This read in part, “This is a cease and desist demand to the supporters of the OpenTofu project. Specifically, OpenTofu has repeatedly taken code HashiCorp provided only under the Business Software License (BSL) and used it in a manner that violates those license terms and HashiCorp’s intellectual property rights. In at least some instances, OpenTofu has incorrectly re-labeled HashiCorp’s code to make it appear as if it was made available by HashiCorp originally under a different license.”

The order continued, “Therefore, we demand that OpenTofu … cease and desist from further violations of HashiCorp’s BSL license and infringement of HashiCorp’s copyrights. If OpenTofu does not comply, we reserve all rights, including the right to send DMCA takedown notices to Github or any other third-party hosting or source code repository provider, and the right to initiate litigation to stop further violations.”

OpenTofu replied, “The OpenTofu team vehemently disagrees with any suggestion that it misappropriated, mis-sourced, or otherwise misused HashiCorp’s BSL code. All such statements have zero basis in facts.” In addition, it said, HashiCorp’s claims of copyright infringement are completely unsubstantiated.

Where’d the Code Come From?

As for the code in question, OpenTofu claims it can clearly be shown to have been copied from older code under the Mozilla Public License (MPL) 2.0. “HashiCorp seems to have copied the same code itself when they implemented their version of this feature. All of this is easily visible in our detailed SCO analysis, as well as their own comments.”

In a detailed source code origination (SCO) examination of the problematic source code, OpenTofu stated that HashiCorp was mistaken. “We believe that this is just a case of a misunderstanding where the code came from.” OpenTofu maintains the code was originally licensed under the MPL, not the BSL. If so, then OpenTofu was perfectly within its right to use the code in its codebase.

OpenTofu’s attorney said, “Both the OpenTofu files to which you refer and HashiCorp’s Terraform files to which you compare them are both derived (at least to some degree) from the pre-fork MPL-2.0 files – code that was made publicly available under the MPL.” Therefore, “To my client’s knowledge, none of the Terraform code subject to the BUSL has been improperly copied, incorrectly sourced, or used for any purpose.”

The attorney went on to say that HashiCorp should have known this. “[W]hile you complain that a comparison of HashiCorp Terraform code to OpenTofu files show ‘substantial similarity’ between the two, the diff files you attach to your Letter actually show the compared files to be quite dissimilar.”

The OpenTofu lawyer continued, “Going forward, we would be open to establishing developer liaison contacts between OpenTofu and Terraform, who would be available to review and address any intellectual property concerns with submitted contributions to OpenTofu or Terraform.”

Laferra concluded, “In the future, if you should have any concerns or questions about how source code in OpenTofu is developed, we would ask that you contact us first. Immediately issuing DMCA takedown notices and igniting salacious negative press articles is not the most helpful path to resolving concerns like this.”

So, what’s next? Having covered more than my fair share of intellectual property legal conflicts, I don’t expect HashiCorp to back down. I fear that this issue will be resolved in the courts, not in Git repositories.

Photo credit: Maarten van den Heuvel on Unsplash

April 11, 2024
Hashicorp Versus OpenTofu Gets Ugly

When HashiCorp dumped its open-source Mozilla license for the Business Source License (BSL 1.1) for its flagship program, infrastructure as code (IaC) tool Terraform, many Terraform developers and partners were ticked off. As open source developers are wont to do in such situations, they created a Terraform fork, called OpenTofu.

So far, that’s business as usual when a company shifts a project away from open source. Things heated up, though, when the Linux Foundation announced it would support the new fork. Now, the conflict between HashiCorp and OpenTofu is starting to boil, as HashiCorp accused OpenTofu of “not respecting the terms of its BSL license governing its Terraform codebase.”

Them’s fighting words!

OpenTofu replied, “OpenTofu vehemently disagrees with any suggestion that it misappropriated, mis-sourced, or otherwise misused HashiCorp’s BSL code.” The organization added, “HashiCorp may be conflating code that it had previously been open-sourced under the MPL and more recently developed code it published under the BSL.” And, OpenTofu says, it will “issue a written response providing a more detailed explanation of its position.”

So, what is this code in question? Neither Hashicorp nor Terraform is saying, quite yet. However, it appears to be a “removed block” automation process that enables Terraform to manage resource deletion.

Matt Asay, MongoDB‘s VP of Developer Relations, revealed this accusation. He accused OpenTofu of lifting the “removed block” code, which was released under the BSL a few months after the OpenTofu fork was created. “The tell? OpenTofu took this BUSL-licensed HashiCorp code, removed the headers, and tried to instead relicense it under the Mozilla Public License (MPL 2.0),” Asay wrote.

Oh, it’s on now.

Even before news of the legally threatening message emerged, prominent open source leaders attacked Asay’s claim. Bryan Cantrill, the hyper-cloud Oxide Computer co-founder and CTO, claimed, “This is an extraordinarily serious accusation – and the cited files frankly don’t substantiate it.”

Cantrill is far from alone in thinking that there is insufficient evidence to justify the accusation that OpenTofu has stolen any code. Until we have harder evidence, the issue will remain a case of “He said, she said.”

Chris Aniszczyk, the CTO of the Cloud Native Computing Foundation (CNCF), a Linux Foundation-related organization, doesn’t think it’s a good look for HashiCorp. He said it’s “embarrassing to see a company light all of its hard-earned developer reputation on fire, on top of attacking open source.”

Dan Lorenc, CEO and co-founder of Chainguard, a software supply chain security company, warns this could be the start of something big and ugly in software development circles. “I don’t have any philosophical or ethical issues with source-available licenses, but their proponents are going to have to address these cease-and-desist threats,” he wrote. “Developers working in OSS (OSI/FSF approved licenses) are going to be prevented from engaging with or even looking at source-available projects completely to prevent the risk of lawsuits. It’s just too risky to pretend otherwise.”

Lorenc adds, “There’s no gradual transition here, it’s going to be a hard fork in communities if these lawsuit threats keep continuing. And I know which community I’d bet on sticking around.”

I lived through the SCO-Linux copyright wars. I think I know who will win, too.

Photo credit: Quino Al on Unsplash

April 9, 2024
Rebellion Against Changes to Open Source Terraform License Mounts

A segment of the community that has embraced open source Terraform software, originally developed by HashiCorp to provision infrastructure as code (IaC), has published a manifesto calling on the company to reverse recent changes made to licenses that determine how the software can be used.

If that demand is not met, the group plans to launch its own fork dubbed OpenTF under the terms of the previous license that governed the usage of Terraform.

Pawel Hytry, CEO of Spacelift, a provider of a platform for orchestrating the management of multiple IaC tools, said the issue comes down to restrictions that Hashicorp has put in place restricting use within platforms that have embedded it, in addition to limitations on how Terraform can be used in production environments.

HashiCorp is shifting from a Mozilla Public License v2.0 (MPL 2.0) to a Business Source License (BSL) v1.1 in all future releases of its software including Terraform, Vault, Consul, Boundary, Nomad, Waypoint, Packer and Vagrant, as part of an effort to maintain maximum control over its intellectual property.

This shift will have a chilling effect on Terraform innovation because fewer individuals and organizations will want to contribute to advancing the tool, said Hytry.

However, Hytry added, historically HashiCorp has not been perceived as not being overly receptive to contributions from external developers, so the bulk of the code used to create Terraform has come from developers working for HashiCorp.

It’s not clear to what degree this firestorm impacts DevOps teams, but the fragmentation of open source projects stemming from issues that come down to revenue streams is increasing as it becomes more difficult for vendors that create open source software to continue to increase the number of subscription services for the support they provide. Red Hat, for example, recently decided to no longer make the source code for the Red Hat Enterprise Linux (RHEL) edition of its operating system available. The decision led to Oracle, SUSE and CIQ pooling their resources to create a fork of RHEL that they promise will be compatible with future updates to RHEL.

It remains to be seen how many vendors might throw their weight behind OpenTF. HashiCorp, in addition to having partners that built the business on the open source edition of Terraform, also has plenty of rivals that offer IaC tools.

It may be a while before this conflict is resolved, but for now, the situation remains fluid, said Hytry. DevOps teams that make extensive use of Terraform will need to monitor events to make sure any future usage of Terraform complies with BSL terms that supporters of the manifesto claim are at this point too vague.

In the meantime, organizations that rely on open source software should brace themselves for any abrupt changes to licensing terms that may be forthcoming in more challenging economic times. In fact, one of the hidden costs of using open source software is the different terms and conditions under which it is licensed. Resources to monitor each additional instance of open source software for compliance, with terms and conditions that are not always as simple or clear cut, need to be dedicated to a task that doesn’t make any meaningful contribution to return on investment.

August 18, 2023
Pulumi Embeds Terraform Conversion Tool in Platform

At its virtual PulumiUP event, Pulumi this week announced it embedded an ability to convert instances of Terraform into its infrastructure-as-code (IaC) platform via a simple command line interface (CLI) command. In addition, Pulumi added a Review Stacks capability that automatically creates a dedicated cloud environment to review code every time a pull request is opened.

Pulumi is also making available a beta release of Pulumi Azure Native Provider 2.0, which decreases the size of the software development kit (SDK) by 60%. This is part of an effort to improve deployment times and reduce resource consumption by, for example, removing support for deprecated application programming interfaces (APIs).

Finally, Pulumi has added support for property names and values to Resource Search for Pulumi Cloud, a search and analytics tool for cloud computing environments. DevOps teams can now additionally search for instances of, for example, versions of databases or specific availability zone as a result.

Aaron Kao, vice president of marketing for Pulumi, said previously the company made available a separate tool for converting snippets of Terraform code. For the first time, Pulumi is also making available a converter plugin in the Pulumi engine which allows conversion tools from other IaC tools to be used.

Review Stacks, meanwhile, makes it simpler for DevOps teams to pick up an unfamiliar codebase, make changes to both application and infrastructure code and share a live environment to review that code.

Pulumi has been making a case for an alternative to Terraform that centralizes the management and provisioning of infrastructure using a set of embedded guardrails. One of the major issues enterprise IT organizations are encountering these days is the number of misconfigurations created when developers use tools like Terraform to provision infrastructure on their own. Those misconfigurations often lead to, for example, ports being left open and cybercriminals exfiltrating data.

That approach is gaining traction among enterprise customers that are looking to centralize the management of DevOps processes, said Kao.

In the wake of a series of high-profile breaches, more organizations are now reviewing the security of their software supply chains, including how infrastructure is provisioned. Many of those organizations will need to revisit those processes as pending legislation that requires more stringent requirements for building software eventually become law, noted Kao.

At the same time, many organizations are also looking to improve the productivity of DevOps teams during an economic downturn that limits the number of full-time employees they might be able to hire. The pace at which software is being developed has not slowed, so there is a greater urgency to do more with less.

It’s not clear how quickly organizations are centralizing the management of DevOps workflows, but with the rise of platform engineering, there is clearly more interest in defining consistent backend processes that multiple DevOps teams are currently managing individually. The challenge is finding a way to achieve that goal without impeding the pace of innovation enabled by embracing DevOps in the first place.

June 15, 2023