devopsy.kinsta.cloud

Category: IT as Code

FinOps Cloud Cost Optimization via Governance-as-Code
Cloud implementations continue to grow in scope and complexity. Too often, this expansion also means costs keep spiraling out of control. In this article, we’ll examine why the emerging practice of cloud financial management (FinOps) is so essential to containing costs in the cloud. We’ll then discuss the challenges teams encounter as they start to apply FinOps approaches. Finally, we’ll reveal the opportunity presented by employing governance-as-code to establish continuous cloud cost optimization—and how this approach makes it practical for teams to put the theory of FinOps into practice so they can keep cloud costs in check.

The Emergence of FinOps

For many organizations, there’s been a race to leverage cloud services, with teams seeking to realize advantages in flexibility, agility and efficiency. Within an organization, it’s not only a matter of more workloads running in the cloud, it’s a matter of more distinct deployment models being used, more teams using cloud services and leveraging multiple service providers.

In the process, teams have had to contend with rapid growth in the size and diversity of the implementations that have to be managed—and paid for. It is this latter aspect that has been a rude awakening to those responsible for managing budgets. Left unchecked, this growth in cloud use has led to spiraling costs and predicting, managing and controlling these costs only grows more difficult as deployments get more complex and more frequent.

This need for improved cost management has given rise to a new discipline: Cloud financial management; or FinOps. In short, FinOps is a way of bringing financial discipline and accountability to the variable-cost paradigm found in the cloud. FinOps brings together a combination of technologies, processes and teams, including professionals from technology, finance and business. By gaining an improved ability to understand, track and manage cloud costs, teams can begin to maximize the value realized through their cloud expenditures. Given the strategic and urgent nature of cloud cost management, it’s not surprising that FinOps teams are growing. According to research conducted by the FinOps Foundation, FinOps teams expanded 47% over last year, and these professionals are expecting their teams to grow 75% in the next year.

Challenges in Establishing FinOps

While there’s a lot of promise behind the establishment of FinOps practices, there are also many obstacles standing in the way for teams. When the FinOps Foundation queried respondents about top challenges, these were the top five responses:
- Getting engineers to take action (39%)
- Dealing with shared costs (33%)
- Accurate forecasting (26%)
- Reducing waste or unused resources (25%)
- Full allocation of costs (23%)
Lacking comprehensive control and visibility over multi-cloud environments means these teams can’t proactively manage costs—and too often find out about wasted expenditures and high costs of underutilized resources after the fact. Because stretched teams are reliant upon manual follow-ups and tactics, it is difficult to get engineers to respond as quickly and effectively as needed.

Compounding these challenges is the fact that across teams, different tools and processes are used. For example, different development teams might be using different provider services along with varying deployment tools, such as AWS Cloud Formation, HashiCorp Terraform, Kubernetes and so on. This inconsistency introduces inefficiencies and miscommunication and stifles team productivity.

While the cloud offers a range of benefits compared to legacy, on-premises infrastructure, without proper governance, cloud costs can quickly spiral out of control. In fact, Gartner analysts report that in the next few years, 60% of organizations will encounter cost overruns in public clouds.

Rethinking Cost Management

For all the reasons articulated above, teams are being asked to take a new approach to cloud cost management, one that aligns with the principles of FinOps. For many, this means embracing the concept of governance-as-code to optimize cloud costs. Cloud governance-as-code represents a way to automatically and dynamically apply policies, including for cost optimization, in fast-changing cloud environments.

With governance-as-code approaches, policies are defined using a standard language. Code associated with cost policies is created, deployed and maintained in much the same way developers manage application or infrastructure code. Developers don’t have to worry about tracking or managing costs. Cost policies are continuously evaluated against current state, utilization and configuration of cloud infrastructure. If any violations occur, automated notifications and workflows can be triggered. Here are the key requirements for using governance-as-code to establish continuous cloud cost optimization:

Cost Policies are Defined via an Intuitive Language

Traditionally, development, operations and financial groups worked in silos. Each team had its own tools, workflows and policies, and even fundamental terminology could vary substantially.

To maximize the potential of continuous cloud cost optimization via governance-as-code, solutions need to enable teams to define, manage and revise policies using an easy-to-understand, consistent language. This makes it much easier for different teams to collaborate and gain alignment on key cost management policies and best practices.

Finally, to be effective in establishing continuous cloud cost optimization, governance-as-code offerings need to employ a language that is independent of any specific programming languages that application developers may use. Further, these languages should be applicable to the broadest set of applications, cloud services, development tools and so on. This is vital to ensure that development teams retain maximum flexibility in the tools and services they can use.

Cost Policies are Managed and Deployed via Git Within CI/CD Workflows

Once policies are reviewed and finalized, teams must be able to apply these policies within the development organization’s existing continuous integration/continuous delivery (CI/CD) workflows.

Through robust governance-as-code approaches, teams can create cost policies as code artifacts and deploy them via open source version control systems like Git. Through this approach, teams can apply policies in a manner consistent with their software development and delivery life cycles.

By codifying cost governance in this way, cost controls can be deployed via Git within a CI/CD pipeline. This approach enables teams to avoid complex manual processes associated with tasks like managing tickets, requesting changes and so on. This also helps teams keep up with the accelerating pace of today’s dynamic cloud and development environments. Further, this approach means governance code can be traced through various state changes, rolled back if needed and so on, just like application code.

Cost Policy Violations are Detected and Addressed in Real-Time

Through continuous cloud cost optimization via governance-as-code, teams are empowered to institute not only continuous detection but also immediate action. Policies can be employed at regularly scheduled intervals or they can be triggered based on certain events. This approach enables enforcement of workflows and actions that may be required. If policy violations occur, notifications to relevant teams or team members can happen automatically and advanced workflows can be triggered. For example, a policy could identify RDS instances greater than 30 days old that have had no connections in the last 14 days. It would then notify the owners of these instances. If they don’t take action in seven days, the instances could automatically be stopped.

Cost Policy is Continuously Enforced and Refined

Continuous cloud cost governance-as-code has a never-ending life cycle, just like the team’s software development life cycle. Cost governance code is built, deployed, tested and released, and the process continues as new policies are developed and refined over time. As requirements change, teams need to be able to periodically analyze cost data and update policies as needed.

As organizations continue to invest larger sums in their cloud implementations, the need to establish strong FinOps capabilities will only grow more critical. However, as these cloud deployments continue to grow more complex, it can get increasingly difficult to realize the promise of FinOps. For these reasons, employing governance-as-code to establish continuous cost management is emerging as a key approach. With this approach, teams can achieve sustained compliance and minimize waste.
October 22, 2021
HashiCorp Previews Updates to Consul Service Mesh

During an online HashiConf Global conference, HashiCorp today made available a beta of an update to its open source Consul service mesh for virtual machines and Kubernetes clusters that makes it possible for multiple administrators to share the same instance of a multitenant service mesh.

Version 1.11 of Consul will also add support for a command line interface (CLI) for Kubernetes that will provide an alternative to Helm charts for installing Consul on those clusters.

Raymond Austin, director of product marketing for HashiCorp, said it’s become apparent that there is a need for a higher level of abstraction for managing application programming interfaces (APIs) that is easy to deploy across both legacy IT environments based on virtual machines and cloud-native environments based on Kubernetes clusters.

Of course, in many cases, those Kubernetes clusters are deployed on top of virtual machines. However, there are also now more instances of Kubernetes being deployed on bare metal platforms at, for example, the network edge.

As IT environments become more complex, a service mesh makes it simpler to discover, access and secure APIs that are now rapidly proliferating across the enterprise. That transition requires a more dynamic approach to not only managing APIs but also the network services those APIs depend on, added Austin.

HashiCorp is betting that many IT organizations will prefer to rely on a lightweight open source platform supported by a single vendor rather than relying on more complex open source service mesh offerings being developed under the auspices of a consortium.

It’s still not quite clear who within an IT organization is assuming responsibility for deploying a service mesh. Most often, it’s a development team that makes the initial decision. However, as IT operations teams become more aware that service meshes represent a capability that needs to be horizontally made available across multiple applications, it may be only a matter of time before some organizations decide to standardize on a single service mesh to reduce operational overhead, noted Austin.

Service meshes will also play a critical role in simplifying management of networking and security services at a higher level of abstraction within a larger DevOps workflow.

It’s still early days as far as service mesh adoption in the enterprise is concerned, but there are already a large number of service mesh options. In fact, there may never be a single dominant service mesh platform. Interoperability standards still need to be ratified that enable organizations that have standardized on different service meshes to apply policies to microservices spanning hybrid cloud computing environments.

Of course, not every IT organization requires a service mesh yet, either. Many IT teams can still get by using a traditional proxy server or an API gateway until the number of microservices-based applications deployed starts to achieve critical mass. One way or another, however, the more APIs there are to manage, the more inevitable it becomes that an IT team will deploy a service mesh.

October 20, 2021
Pulumi Adds Registry to Share Secure IaC Code

Pulumi this week added a public registry to its portfolio that makes it easier to discover packages that have been created using its tools to manage infrastructure-as-code (IaC).

Joe Duffy, Pulumi CEO, said the Pulumi Registry will make it simpler for DevOps teams to share and reuse IaC tools that have been vetted versus always requiring each development team to write new code each time they want to programmatically provision infrastructure. Pulumi Packages provide modern cloud reference architectures in the form of software development kits (SDKs), code samples and how-to guides.

One of the primary reasons that security has become a major cloud issue is that developers often misconfigure infrastructure. The registry provides a way to reduce those configuration errors by making it easier for developers to reuse code written in TypeScript/JavaScript, Python, Go and .NET that they have stored in repositories such as npm.

In addition to the major cloud service providers, there are also Pulumi Provider Packages for Auth0, CloudFlare, Confluent Cloud, Datadog, DigitalOcean, Docker, GitHub, Kong, MinIO, MongoDB Atlas, PagerDuty, Snowflake, Spot by NetApp and others.

There are also Pulumi Component Packages for deploying container applications on instances of Kubernetes and other related platforms in addition to deploying applications on serverless computing frameworks.

Duffy said in addition to providing a “golden image” of their configurations the registry reduces the overall level of friction that often exists between developers, IT operations teams and cybersecurity professionals.

The latest Pulumi offering is part of an ongoing effort to democratize cloud computing in a way that is repeatable and reliable, noted Duffy. Pulumi, for example, just added support for a set of AWS Cloud Control application programming interfaces (APIs) that reduce the number of APIs that developers would otherwise have to master. The more APIs there are, the more likely it is a developer will make a mistake.

IT teams should expect that other cloud service providers will similarly streamline the APIs they present developers as part of an effort to make their services both more accessible and secure. In the meantime, frameworks provided by Pulumi provide a means to invoke those lower-level APIs at a higher level of abstraction for both developers and internal IT operations teams alike.

It’s not clear today how many developers routinely employ IaC tools, but as the number of workloads deployed in the cloud steadily increases, so too does the percentage of developers that employ these tools. While that shift represents a major boon in productivity for developers, it brings with it additional security risks that many organizations fail to appreciate until it’s too late.

The only way to really solve that issue is for organizations to adopt DevSecOps best practices that require a level of collaboration between DevOps and security teams that is still relatively rare. However, as the tooling provided continues to improve, the technical challenges associated with achieving that goal continue to decline. The next big issue, as always, is bringing together all the diverse cultures within an IT organization to actually implement those tools and define a set of best practices that work for all concerned.

October 20, 2021
A Blueprint for Securing Software Development
Software development has changed dramatically in recent years, as technologies like DevOps, application containers, and cloud-native transform how software is built and distributed. Unfortunately, attackers have been paying close attention to these changes, and have retooled their attack strategies to take advantage of relatively weak security controls in software development and build environments. Attackers recognize that security teams are more focused on protecting infrastructure and less focused on software development and supply chains. This is how damaging attacks like SolarWinds happen – and it’s likely that attackers will strike again to yield even more successful outcomes.

A new approach is needed to secure the software development pipeline. What’s more, this approach needs to be put in place by the engineers developing the software—and not by information security teams. Engineering teams need to be responsible for design and implementation while giving security teams visibility and assurance that controls are operational and that policies are being enforced.

A new blueprint for securing the pipeline centers on a flexible set of controls that secure the software build pipeline by ensuring that authentication and authorization are properly managed, that the integrity of software artifacts are tested at appropriate stages and that controls are placed on third-party and open source solutions incorporated into the software. The result: Software innovation and delivery at lower risk of supply chain tampering, and reduction of attacks during development.

A Pragmatic Design Philosophy

This modern software security blueprint is based on the following design philosophy:
- Least-privilege: Only grant the access and permissions required to accomplish a job, and no more.
- Immutability: Artifacts and infrastructure are not modified after deployment to an environment. If a change is required, it’s done in the image or script in the development environment and then promoted through the higher environments.
- Everything-as-code: Infrastructure, security policies and other parts of the pipeline are also implemented as code and subject to the same controls as the software artifacts.
- Traceability: All changes, whether to infrastructure or business code, are version-controlled.
Controls Blueprint for Software Development Pipelines

The basis for this software security blueprint centers around a set of 15 controls that are built for modern development pipelines where developers, pipelines and execution can happen anywhere in the world.

Control one: Restrict administrative access to CI/CD tools. Ensure that only authorized people can make administrative changes to the CI/CD system, which prevents unauthorized changes to pipeline definitions.

Control two: Only accept commits signed with a developer GPG key. Unsigned code commits are challenging, if not impossible, to trace and pose a risk to the integrity of the codebase.

Control three: Automation access keys expire automatically. By ensuring that access keys used by automation expire periodically, developers create a shorter window of attack when keys are compromised.

Control four: Reduce automation access to read-only. CI systems should have read-only access to source code repositories following the principle of least-privilege access.

Control five: Only dependencies from trusted registries can be used. By configuring the dependency manager to only allow connections to an authorized list of registries, attacks can be blunted by keeping malicious packages in public registries from entering the pipeline.

Control six: Any critical or high-severity vulnerability breaks the build. Supply chain attacks may introduce code with vulnerabilities into the software pipeline. Static application security testing (SAST) helps to identify serious security issues, including poor cryptographic practices, hard-coded credentials and injection vulnerabilities.

Control seven: Artifacts are stored in a repository in development, staging and production. This control helps to enforce the immutability of the artifacts so they can be compared in the development, staging and production repositories.

Control eight: Validate artifact digest. Before an artifact is deployed in any environment, its digest should be validated against the artifact in the repository to ensure that it has not been compromised.

Control nine: Pull requests require two reviewers (including one default reviewer) and a passing build to be merged. In addition to supporting good coding practices, this control also helps ensure that no commits are made without competent human oversight.

Control 10: Artifacts in higher repositories are signed. Requiring artifacts to be signed in a repository throughout the process ensures visibility and traceability for whatever is deployed to production.

Control 11: Available container images don’t have any high or critical vulnerabilities. Just as applications need to be tested for vulnerabilities before being placed in production, so do the container images into which they are packaged for deployment. Container images could have open source vulnerabilities if they contain open source software.

Control 12: Validate artifact signatures and digests. Validating the signature of the artifact against the digest ensures that the artifact was not tampered with in the repository and that the artifact being deployed is the same one that was tested.

Control 13: Scan deployed images in production. This helps to ensure that the prior controls were followed for all software in production.

Control 14: Validate Kubernetes resource manifests. If the resource manifests are tampered with, they could be tricked into deploying a container of the attacker’s choice.

Control 15: Ensure build environments are ephemeral and immutable. Build environments should be defined in code with automated creation and teardown and a fresh environment created for every build. Build hosts should not be accessible using interactive logins.

We believe this blueprint provides engineering teams with an actionable architecture that security teams can rally behind and support. By adopting these standards, engineering teams are promised the delivery of assured software at lower risk of supply chain tampering, reduction of attacks during development and manipulation in staging and production.

Here’s what it looks like:

To learn more about the suggested blueprint, read our GitHub post about the project. Our goal is to prioritize speed and agility for software development while incorporating a security-always mindset.
October 19, 2021
Trust in Cloud Providers is a Developer Issue
Earlier this summer, we worked with ClearPath Strategies to ask developers about the level of trust they have in their various cloud service providers. What we learned spoke volumes: While developers’ familiarity and comfort with Amazon, Microsoft and Google position these well-known cloud providers as market frontrunners, developers often do not agree with the companies’ values.

The research project gathered insights from more than 800 developers at small and mid-sized businesses, those with less than 500 full-time employees, across a wide variety of industries and geographies worldwide. You can download the summary report here (no signup required).

In ClearPath Strategies’ report, senior analyst Stephanie Fairchild said that “approximately 20% of developers believe that the big three hyperscalers will engage in behaviors that include leveraging their brands to stifle competition, focusing on shareholders more than customer needs, hosting or working with customers that developers find morally objectionable, monetizing user data for their own use and locking them in with proprietary tools. Criteria such as security and performance clearly take priority over these trust issues for developers, but it’s interesting to note that a sizable portion of developers are thinking about how their values align with those of their service providers.”
- Trust plays an important role in developers’ choice of a cloud hosting provider, but needs take precedence; when prioritizing needs or values, developers report they prioritize needs over their values 73% of the time.
- The top three reasons cited for developers’ preference of their chosen third-party cloud provider: Security, trust and quality of service.
- Survey respondents indicated that both hyperscale and alternative cloud providers deliver equally effective performance and security.
The report cited three top criteria for choosing a third-party cloud provider: Security, trust and quality of service. Survey respondents felt that hyperscale and alternative cloud providers deliver equally effective performance and security. Interestingly, although 73% of respondents said they prioritize needs over values when making a purchasing decision, approximately 20% said they don’t trust the big three providers (AWS, GCP and Azure) in what could be considered key moral areas.

It’s no secret that the extensive functionality and brand awareness of hyperscalers make them popular choices for developers, but an alternative cloud provider like Linode, DigitalOcean, OVHcloud or Hetzner can be a more trustworthy partner for innovative upstarts—one that likely offers better pricing and doesn’t threaten or compete with their business. And when an alternative cloud provider meets a developer’s functional needs—which is nearly always the case—developers can have the best of both worlds: A provider they can trust to meet their needs and one that shares their values.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2021 – October 11-15, 2021
October 15, 2021
Energy Efficiency and the Data Center

The IT industry has historically been a leader in the ongoing race to reduce CO2 emissions. Reducing emissions is good for the environment and for the corporate bottom line. The data center is well-known as a voracious data consumer. But the data center consumption findings keep pace with the trend. Even in the face of remarkable growth in data center capacity and performance, their energy consumption has increased only a few percentage points since the 2010s. This almost-flat rate of energy consumption is primarily the result of the shift to cloud computing as well as the shift to solar and wind energy, where it makes sense.

Investments in renewable energy sources have helped tamp down criticism over the overall IT carbon footprint. Knowing that it takes a tremendous amount of energy to power a data center, owners and operators are always looking for ways to improve efficiency and reduce carbon expenditure. Today, there are newer and better ways for organizations to deliver the performance to properly power modern applications while keeping in mind the energy efficiency and carbon reduction the world requires.

Minimizing Energy Use and CO2 Production

How did we get here? Decades of semiconductor process miniaturization in concurrence with living out Moore’s Law has culminated in ongoing improvements in performance, storage capacity and device density. Today’s mobile phones are a great example of that ongoing progress. A secondary factor in the mobile phone revolution is a processor architecture optimized for efficiency and flexibility. This ARM architecture has been used to power almost all devices since the first iPhone. Over the years, ARM has also become a realistic platform for servers; often a better choice than an x86 system.

Energy efficiency is one area where ARM systems win out over conventional server architecture. But it was not until recently that ARM processors could deliver the required performance for the data center. Today’s systems, based on Armv8 designs, have brought into play x86-class capabilities in a more energy-efficient way. The advent of the ARM-based AWS Graviton processors, which by some estimates have already captured 20% of the instance share on AWS, demonstrates that ARM can run production workloads while delivering what AWS calls “up to 40% better price performance over comparable current-generation x86-based instances for a wide variety of workloads.”

To illustrate the impact on carbon production when a data center shifts from x86-based architecture to ARM servers, an analysis of the top elements contributing to data center energy use was undertaken. The energy savings was calculated and translated to reduced CO2 emissions based on the fuel mix used to power the electric grid in various locations. The results prove that an ARM-powered data center reduces CO2 production by 74%, equivalent to almost half a million barrels of oil.

Power Consumption Reduced by Efficient Servers

The premise is simple—innovative server design significantly reduces data center energy use. In this model, a server’s power draw and physical size are key factors. The energy needed for other data center subsystems was extrapolated by using ratios derived from experience and validated by estimates from other data center operators.

Data center energy use can be divided into five categories of equipment and infrastructure: Servers, network communications, power supplies, conversion and UPS, cooling systems and storage. Servers and cooling comprise about 75% of the total.
The calculations were based on the energy used by a medium-sized data center with 750 racks of conventional 1U servers (42 servers/rack; around 31,500 in total). Assumptions used included servers comprising two-thirds of the floor space and that the equipment space accounts for 65% of the total, translating to a 62,000 square-foot data center. For comparison, facilities operated by the major cloud vendors can be a million square feet or more. Knowing server power consumption and the ratios for the equipment categories, algebra can be used to calculate the energy consumption for each.

The data center in this hypothetical example has, as we said, 31,500 servers. Using the 912.5W power draw of a Dell R640 as typical of 1U x86 servers, the total server usage in the ARM data center would be 251,795 MWh/year. Using this formula, the amount of energy saved by using ARM servers can be calculated. Each server requires about one-quarter the electricity of a standard server. In the formula, the assumption is that the ARM server installation would require half the equipment racks (375 instead of 750) and 52% of the power-per-system, comprising a data center using only 26% as much electricity (152,409 MWh/year) as one based on x86 systems.

The math shows that switching one medium-sized data center to ARM servers can produce annual savings equivalent to 45,459 fewer cars on the road or 486,749 fewer barrels of oil used.

The value in CO2 reduction is widely understood for preserving the environment. It is also advantageous from an economic standpoint. In fact, there are many technical and economic reasons to adopt energy-efficient ARM-powered servers in the next-generation cloud data center.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2021 – October 11-15, 2021

October 15, 2021
CNCF Adopts Meshery to Advance Service Mesh Management

The Cloud Native Computing Foundation (CNCF) announced this week during the ServiceMeshCon/KubeCon + CloudNativeCon conference that Meshery, a service mesh management plane created by Layer5, has become a sandbox-level project.

In addition, Layer5 has also donated Service Mesh Performance (SMP), a set of tools for measuring the efficiency of a service mesh, to the CNCF. SMP provides an open source framework to define standardized benchmarking practices, performance test configurations and measurements as part of an effort to create a MeshMark index for rating a service mesh. A set of service mesh performance methodologies will also be published by the IEEE later this month, developed in collaboration with engineers from Layer5, Intel, Red Hat and HashiCorp.

Layer5 CEO Lee Calcote said the goal is to make it simpler for IT teams to determine which service mesh to employ based on their specific use case requirements.

Several service mesh platforms are starting to gain traction as a means to both manage application programming interfaces (APIs) and provide a layer of abstraction between lower-level networking and security APIs that makes those services more programmatically accessible to developers.

Meshery itself is an implementation of SMP and the conformance tool created for the Service Mesh Interface (SMI), a separate CNCF project to promote interoperability between service meshes. Meshery is based on a set of microservices that are designed to make the platform easily extensible.

That approach makes it possible to manage multiple instances of service meshes that might be deployed on virtual machines as well as Kubernetes clusters using a range of management tools that are part of the Meshery project.

Meshery supports AWS App Mesh, Citrix Service Mesh, HashiCorp Consul, Istio, Kuma, Linkerd, Network Service Mesh, NGINX Service Mesh, Open Service Mesh and Traefik Mesh. Meshery also provides IT teams with more than 60+ best practices in the form of templates that can be used to deploy a service mesh.

It’s still early days as far as adoption of service mesh in the enterprise is concerned. Most IT teams still rely on either proxy software or a gateway to manage APIs. However, as the number of APIs that need to be managed steadily increases, it won’t be long before many IT teams start to at least evaluate a service mesh.

In the longer term, the impact those service meshes are likely to have on the management of IT could be profound. It is becoming apparent that a service mesh also provides a higher level of abstraction for invoking networking and security services. In fact, the service mesh may wind up playing a key role in driving the convergence of network and security operations with DevOps best practices.

In the meantime, IT teams will need to decide to what degree they may want to standardize on a single service mesh versus letting individual teams implement their own preferred platform that could one day be centrally managed via Mashery. One way or another, the way APIs are managed and invoked will soon fundamentally change forever.

October 14, 2021
Top 5 Must-Haves for IaC Automation Tools

These days, there are a lot of different DevOps tools to accomplish a lot of different jobs. Almost daily another startup comes out with a new and innovative product or a newer (maybe even better) version of existing tools. One of the biggest innovations has been infrastructure-as-code (IaC). Giving infrastructure admins and developers alike the ability to create, manipulate and destroy infrastructure using code files changes the way we administer our environments. Though these tools are revolutionary and can solve many problems, sometimes they come with a whole new set of issues—as can any type of tooling, really.

Generally, IaC tools are run locally at first. When an infrastructure admin or developer is just starting out with it, they store the files locally; maybe in a repository, then clone the files to execute locally. This is perfectly acceptable and a very common practice. The issues arise when it comes to scaling this practice to a team; for example, with regard to visibility and governance. If everyone is running their deployments locally, the team doesn’t have clear visibility into who is deploying what, when and with what variables. This is where IaC automation tools come into play.

There are many different types of tools to solve these problems. Some are better than others. CI/CD pipeline tools, custom-built automation platforms, purpose-built IaC automation platforms—each has their pros and cons. Getting into all of that can be overwhelming and we could go on for hours about each option. But, we’re here to talk a bit more generically today. So, we’ll save all of that for another time. For now, let’s take a step back and talk about the top five must-haves for any infrastructure-as-code automation platform you’re considering. Because each company and use case is different, we’ll talk about these considerations in no particular order.

Role-Based Access Control (RBAC)

This first factor is pretty important. I know I said I wasn’t discussing these in any real order, but this one should really be close to the top of the list for anyone looking into IaC automation. We talked about how automating IaC can cause visibility issues when you scale across a team. You may want to know who deployed what, when, where and why, for example. Usually, the reason you want that kind of visibility is that you would like to control all of these aspects. That way you can help limit waste, budget problems, security concerns and the like. You may want to provide self-service deployment access, but only within certain parameters; maybe only to certain people or teams. Having granular RBAC can help you design a security policy that makes the most sense for your organization and its needs. Some tools have this baked in, some take a bit of work to make it work. The point is that you should have some form of control over your deployment process.

Self-Hosted Agents/Runners

Following the security thread: A lot of these IaC automation tools are SaaS platforms. SaaS can be very secure. But sometimes, for compliance or regulatory reasons, you may want to retain a bit more control over your deployments. This is where self-hosted agents or runners come in. This type of design allows you to keep your “secrets”—such as your cloud credentials and other sensitive variable information—secure in your own way. You can use whatever secrets management solution you want, be it something from your cloud provider or another third-party solution. It also allows you to keep your code secure. All of these tools, when deploying your code, have to get a copy of that code to deploy. So, they’ll do a ‘Git clone’ or some kind of file copy process to get your files from where they are stored, to execute them. If this is a SaaS solution, then they will essentially have access to your code. Again, this may not be a huge issue for some, but it very much is for others. If you don’t want to give a third party this level of access to your code and secrets, a self-hosted agent or runner can help keep it all secure and under your control.

Plan on Pull Request

This feature is a big one when it comes to workflow. Some may not even know how important plan on pull request (PR) is until they realize how easy it can make their life. This feature may be called something different depending on the platform. Plan on PR, PR plan, speculative plan, etc. Essentially, this means that whenever you open a pull request against a branch, your automation platform will complete a deployment just up to the “plan” phase so that you can see exactly what this code change will do. For example, if you accidentally add an extra 0 to a variable or code file, you may end up with 100 instances instead of 10. Ideally, you’d like to know what was going to happen with a change before it is deployed, giving you the option to quickly fix a potential mistake before it actually goes through. In DevOps, you always want to ‘fail faster’ and close the loop so that big mistakes are caught before they happen. That is why this feature is so important. And if the automation platform you choose is doing this correctly, when it runs this plan phase it’ll automatically update the pull request directly with the possible changes as a comment. This way, your developers can see what is going to happen before they try to merge the deployment back to the main branch for deployment to production. And if they take it a step further by adding service checks to the pull request, you can even configure it so that if the deployment plan fails for any reason, developers are actually blocked entirely from merging the PR back until it is resolved. All of these features help to enable the ability to work in a GitOps methodology. Don’t get me started on the wildl misuse of the term GitOps—it is very specific. While one of the central pillars of GitOps is using your repository as your “single source of truth” for what should be deployed, it’s not just using Git to store your Terraform files; it is an infrastructure administration via pull request methodology. If you want more information about GitOps, you can read more here.

Continuous Deployment

This feature is essential for most folks. But, depending on the workflow, it can be less important. Even if it isn’t a must-have now, it may be down the road when you evolve and grow into new workflows. Continuous deployment is the ability for your automation platform to have a webhook or other connection to your source code repository so that it can automatically trigger a deployment process whenever you commit or push to the repository. This feature goes hand-in-hand with the plan pull request feature we just spoke about. To make that feature function properly, a CD feature would be needed, as well. Now, I want to make sure I clarify. Just because you have continuous deployment enabled doesn’t mean it is just going to push to production in the middle of the day every time a commit or push is done. You should still be given the ability to pause the deployment process after the plan phase. This way you can validate that the plan looks acceptable before you approve the deployment. For some use cases, like developer sandboxes, you may “auto-approve” the deployments because you don’t care about having to get validation first. It all depends on your workflow and processes. A good platform will give you the ability to control all of this.

Shift Left Extensibility

This one is a bit of a wild card. It can mean a lot of different things depending on the situation. Essentially, what I am talking about here is the ability to integrate—or at least be interoperable—with a multitude of other tools during the deployment process. There is a concept called continuous verification which means to shift some tools and processes left into the deployment process so that you can stay on top of problems before they start. In the PR plan feature section earlier, we talked about catching issues before deploying. This isn’t just for resource configuration issues. What about security issues? Compliance issues? Performance issues? Or even budget issues? If your IaC automation tool gives you the ability to integrate the processes and tools you are using to validate these things into the deployment process, they no longer have to be an afterthought. You never have to worry about going over budget if you are constantly validating the budget each time you deploy. You don’t have to worry about a security issue if you’re validating security during every deployment. This type of functionality can look and be named very differently between tools. But as long as you have the ability to get all of these other groups (security, finance, etc.) involved sooner, everyone will be better off. Like we talked about before, in DevOps, it’s always better to fail faster. And integrating these checks into your deployments can help you achieve this.

Conclusion

We’ve made it through all five! We’ve talked about some basic security features. We talked about some automation workflow must-haves to make your life easier. And we even scratched the surface on some next-step-type things to look into such as continuous verification. That was a lot—hopefully this has been helpful for anyone looking into IaC automation platforms and you now have an idea what to look for. Some features may be named differently, but the ideas behind them are the same. If your platform can’t do these, it may not be the right one for you. Or, maybe it is and that feature just isn’t that important to you. As I mentioned, your mileage may vary. Every organization is different.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2021 – October 11-15, 2021

October 14, 2021
The Inevitability of Multi-Cloud-Native Apps

We all know the story: Enterprises moved to the cloud. They learned about data center virtualization and multi-tenancy and self-service developer provisioning. Along came Kubernetes, and they moved to cloud-native, where they learned about automation, DevOps and infrastructure-as-code (IaC). Now, even though most enterprises are still figuring out how to do cloud-native at scale, there’s a new imperative they must master: Multi-cloud-native applications. And for almost every enterprise with a global footprint, the move to multi-cloud-native applications is not optional—it’s inevitable.

But before we explore why this is the case, let’s first retrace our steps. For almost every enterprise out there, the journey to the cloud started with adopting a public or private self-service infrastructure provisioning and management model. These enterprises quickly realized that this new IT model required a new set of tools to solve challenges they’d never before faced—enabling their developers with self-service provisioning, managing and operating infrastructure. Enterprises figured it out, but as soon as they figured out the cloud, they quickly had to gear up for the second phase: Automating and operationalizing an enterprise-wide rollout of cloud operating models throughout their entire developer organization.

Of course, this phase has its own set of challenges, and a rich ecosystem of tools emerged to address them. Even so, we all can agree that the majority of mainstream enterprises are still in this second phase. They’ve adopted cloud, they’ve expanded its use, and they’re grappling with the challenges around multi-tenancy, security, DevSecOps and governance. Hand-in-hand with this goes a whole new set of challenges around cost management. Suddenly, enterprises are faced with managing the variable costs of public cloud infrastructure: Optimizing compute infrastructure, utilization and capacity to rein in cloud spend. It’s an issue Sarah Wang and Martin Cassado took head-on in their research on the Trillion-Dollar Paradox.

So, enterprises moved to the cloud and are now, for the most part, successfully grappling with scaling it. But nothing stays the same for long, and businesses drive IT to innovate. This is where cloud-native entered the picture to help enterprise software development teams become more agile and help businesses transform to deliver new services and experiences faster. In fact, the consumer experience this model makes possible has rapidly become the norm.

Consistently delivering rapid software iteration across a global footprint forces DevOps organizations to grapple with an entirely new set of technical challenges: Leveraging containerized applications and microservices architectures in production across multiple Kubernetes clusters running in multiple geographies. Customers want an on-demand experience. This third phase is what we call multi-cloud-native, and it was pioneered by hyperscale IaaS players like Google, AWS, Azure and Tencent.

The reality is, of course, that hyperscalers aren’t the only ones who have figured out how to deliver multi-cloud-native apps. Webscale innovators like Doordash, Uber, Twitter and Netflix have done it, too. To get there, they had to make and share their multi-cloud-native apps across every geography where their customers live. And, in turn, to make that happen they had to tackle a new set of challenges: Develop new tools and techniques like geographically distributed, planet-scale databases and analytics engines, application architectures that run apps on the backend close to the consumer in a multi-cloud-native way. This enabled them to do arbitrage across fleets of infrastructure capacity across regions, all while giving their customers that instant experience they increasingly grew to expect.

What enterprises must realize is that this experience consumers have grown to expect from webscalers is rapidly becoming the same experience they expect when accessing their bank account, their insurance apps and their tax records. For almost every planet-scale enterprise, this sets up a whole new set of challenges: To span multiple clouds and multiple geographies running multiple Kubernetes clusters, you now must worry about new ways to do autoscaling, self-healing, disaster recovery, operations management and cost optimization—all of which are things that you’re just now getting figured out for virtual machines (VMs) and clusters in single regions.

Enterprises must face the reality that cloud-native application models are not the destination but rather a waypoint on a journey to becoming multi-cloud-native. Because the consumer experiences delivered by webscalers make this the inevitable destination.

The Enterprise Journey: Cloud, Cloud-Native and Multi-Cloud-Native

Don’t take my word for it.

Uber and Twitter public disclosures describe how the multi-cloud-native model is an integral part of their strategic game plan. Hybrid and multi-cloud are part of this, too, leveraging Kubernetes as a cluster manager, but they still need tools to manage all those clusters across multiple environments in multiple regions at planet scale.

The bottom line is that the architecture that webscale companies are using today is the same one enterprises will need to adopt tomorrow to remain competitive. Alternatively, these enterprises will cede market share and relevance to the next webscale innovator to disrupt an industry. In adopting this model of building multi-cloud-native apps, enterprises will have to deal with unified access management, controlling multiple clusters, resource management and more, all across multiple regions and clusters of capacity, aggregating versus disaggregating cluster resources, using logical services that span clouds versus namespaces that span a single cluster.

The Enterprise Solution: Planet-Scale Cluster Manager

They will need a planet-scale cluster manager to do this; something that can observe costs, operations, performance and SLAs across clusters, across public and private clouds, spanning VMs and Kubernetes through a single pane of glass so they can operate above the clouds in an automated, easy-to-use way. Band-aid solutions like running multiple spreadsheet models to figure out how to optimize are not scalable. What’s needed are tools like machine learning to automate and optimize at scale, bin packing technology to intelligently and automatically optimize across a planet-scale infrastructure fleet and high levels of automation either with humans in or out of the loop. That’s how hyperscalers and webscalers are already doing it.

Every planet-scale enterprise is multi-cloud-native, running data-driven apps. It’s not a fiction: It’s already happening across webscale companies, not just hyperscalers. Multi-cloud-native is the path that every enterprise is going to need to follow if they want to transform themselves to compete with the next webscale upstart. The good news is, the pioneers have left us a map to follow.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2021 – October 11-15, 2021

October 12, 2021
Mastering API Gateways for the Modern Cloud-Native Enterprise

As modern enterprises continue migrating to the cloud and the need for connected applications and devices dramatically increases, API-driven architectures are being widely adopted. In addition to the convenience of being able to efficiently connect and share resources, organizations benefit from the innovation and accelerated development speed that APIs bring in multi-cloud environments. APIs are not new, but as more enterprises adopt modern application architectures, they are increasingly occupying a large amount of a software development team’s delivery efforts.

When enterprises modernize and migrate to the cloud, it’s natural that challenges arise which need to be tackled before they become pain points—especially when it involves a complex microservices environment. API gateways are an essential ingredient that helps businesses overcome many of the challenges associated with APIs. API gateways are deployed at the edge of an infrastructure, responsible for routing client API requests to backend microservices. Modern API gateways also come with additional perks organizations can benefit from as teams and infrastructure grow simultaneously.

Let’s take an in-depth look at some of the major challenges organizations face around APIs, and how to maximize their benefits.

Rapid Adoption Meets Growing Pains

API-oriented architectures greatly accelerate the agility of an IT department, as they’ve become a well-known pattern for modern application architectures. However, this approach can be deceptive, as many critical issues and concerns around APIs aren’t always apparent from the onset. If they are not properly identified, they become growing pains teams have to face on a daily basis.

The first challenge is the need to authenticate users when they make requests. API gateways allow organizations to integrate and consolidate authentication providers using their protocol(s) of choice. With billing and analytics, organizations need to understand the number of resources consumed when processing a transaction and how their customers consume their APIs. API gateways become the natural point at which to collect this data.

In terms of DevOps initiatives, some API gateways use service discovery to configure themselves dynamically which greatly accelerates a development and ops team’s velocity when shipping new features or fixes. To help teams deliver a highly available and resilient platform, API gateways are enablers that provide them with the puzzle pieces for advanced load balancing, rate-limiting, circuit-breakers, mirroring and more. These components make the DevOps mindset a reality. Teams can scale them as needed to realize production-grade environments, continuously updating deployments with software changes and avoiding any unintentional single points of failure (SPOFs).

In recent years, the evolution of API gateways has solved growing pains in the cloud, helping organizations leverage cloud-native technologies and workflows. Combining cloud-native technologies with an API-led integration platform creates an effective architecture for a digital enterprise to increase productivity. Automating production, operation and services is no longer a goal but a must-have.

Modern API gateways features within this environment go well beyond what one would expect at first glance. API gateways can be the cornerstone of a deployment strategy transformation. With hundreds—if not thousands—of services to continuously deploy, the risk of introducing a regression to the platform soars, and being able to address this problem immediately is not enough. API gateways can help you transition to modern deployment strategies and blue/green or canary deployment becomes commonplace—helping deployments become an easily managed daily task.

API Gateways to the Rescue for Cloud-Native Architectures

As part of the shift to modern application architectures, enterprises should invest resources in implementing API strategies as part of their software development initiatives. In turn, they should leverage API gateways to alleviate growing pains and simplify delivery efforts. APIs should be secured, managed, observed and even monetized.

API gateways can also help with critical issues related to authentication, security and SSL, billing and analytics, life cycle management, high availability and resilience, and they can even help you initiate a culture shift toward a DevOps mindset. By combining API-driven architectures and cloud-native technologies, organizations emerge with an effective architecture for the digital era. API gateways help overcome the challenges and unfold the networking intricacies of today’s cloud-centered world.

October 12, 2021