devopsy.kinsta.cloud

Category: Infrastructure/Networking

AlmaLinux Introduces Engineering Steering Committee to Enhance Community Collaboration
The AlmaLinux OS Foundation has announced the formation of the AlmaLinux Engineering Steering Committee (ALESCo) to bolster community engagement and technical governance. This new body aims to guide the technical direction of the AlmaLinux CentOS-clone Linux distribution, ensuring its robustness and long-term stability.

This move came only weeks after AlmaLinux released a new version, AlmaLinux 9.4. This latest release features compatibility with Red Hat Enterprise Linux (RHEL) 9.4 and support for legacy hardware, which Red Hat will no longer support.

Why this move? Cody Robertson, CTO at Hawk Host, a web hosting reseller, said, “This committee provides a structured platform to streamline the engineering capabilities of everyone involved in the AlmaLinux project. By fostering an environment where transparency is paramount, we’re not only engaging engineers but also welcoming non-engineers on equal footing to be a core part of AlmaLinux’s explosive growth.”

Dubbed the “air traffic control” for engineering matters, ALESCo will oversee and guide technical decisions in collaboration with various Special Interest Groups (SIGs). The committee was proposed by infrastructure team members Jonathan Wright, AlmaLiunx’s infrastructure lead, and Robertson. It will be responsible for:
- Technical oversight of the operating system
- Transparency in engineering decisions
- Keeping a focus on long-term stability
- Release management
- Supporting SIGs
What that means for DevOps is that AlmaLinux should—should—be both better and a bit more flexible to its user community than Red Hat has been. AlmaLinux has already shown that it’s willing to step away from RHEL to improve security and provide support for older hardware. Hopefully, this move will help them to add other improvements.

The AlmaLinux OS Foundation board appointed the initial members of ALESCo. It will subsequently internally appoint a chair and new members every six months. This approach aims to foster diverse leadership. In this context, diverse means AlmaLinux wants to listen to its customers and other contributors, such as tech writers and developers. AlmaLinux also wants to keep its Linux community cred by preventing any single point of prolonged control. To ensure decision-making transparency. All ALESCo meetings will be public, with member affiliations disclosed.

Benny Vasquez, the AlmaLinux OS Foundation chair, underlined the commitment to transparency and community involvement, stating, “The creation of ALESCo reinforces our dedication to transparency while opening the door for broader technical input from our community. We are excited about the increased engagement this will bring and appreciate the contributions from Wright, Robertson, and other key proposal contributors like Farrell, Alex Iribarren of CERN, and Daniel Pearson of KnownHost.”

With ALESCo in place, AlmaLinux is poised to enhance its collaborative efforts, ensuring a robust and reliable operating system that meets the needs of its growing community.
May 23, 2024
Defining DevSecOps for IoT

The Internet of Things (IoT) has brought plenty of benefits to companies and consumers, allowing businesses to track inventory in their supply chain, landlords to measure their tenants’ energy consumption, and pet owners to find a lost cat. It’s become a significant part of the Internet infrastructure, too. The number of global IoT connections grew by 18% in 2022 to 14.3 billion active IoT endpoints, with no end in sight.

But for all the technology’s advantages, it also presents new security challenges. IoT technology uses network connections to facilitate communication between devices and systems. When devices are IoT-enabled, they can collect and transmit data in real-time, enhancing efficiency, safety and convenience across various applications and industries. All that enables great technological feats… and gives bad actors new opportunities.

IoT device scale and diversity: Each device, from smart refrigerators to industrial sensors, operates with different technical architectures and security capabilities. This diversity makes it difficult to implement a one-size-fits-all security strategy. The sheer number of devices connected to the Internet makes it challenging to secure every endpoint against potential breaches.

Proliferation of security vulnerabilities: IoT security vulnerabilities typically arise from inadequate security design and practices. Many IoT devices lack robust security features due to cost constraints or oversight, leaving them susceptible to attacks. Devices could be shipped with default, easy-to-guess passwords, or without the capability to be updated or patched, making them easy targets for hackers.

IoT ecosystem complexity: IoT ecosystems typically involve multiple interconnected devices and systems, each adding layers of complexity in terms of management and security. Managing these systems requires careful coordination and a clear understanding of how information flows and is processed within the network. This complexity often results in security being an afterthought or being inconsistently applied.

Decentralized network devices: As security threats evolve, IoT software must adapt through regular updates to address new vulnerabilities. However, the decentralized and widespread nature of IoT devices complicates the updates’ delivery. Ensuring all devices are consistently updated is challenging when devices are geographically dispersed or lack remote update capabilities.

Enter DevSecOps

Most organizations that are cognizant of security are familiar with the concept of DevSecOps and likely employ people in that role. DevSecOps introduces a security-first mindset to the DevOps process, ensuring that security and compliance measures are integrated from the start. The main objective is to create a collaborative environment where the security, development, and operations teams work together to address security issues, reducing the potential risks and vulnerabilities.

As in other software development projects, it’s important for programmers and system designers to work with the security team and DevSecOps in particular.

Incorporate automated security testing tools into the CI/CD pipeline: This helps address security throughout the IoT development process. These tools automatically scan for security vulnerabilities in the code from the earliest stages of development through to deployment, making it easier to identify issues before production.

Ensure all IoT devices are securely provisioned: By securing these devices from the point of manufacture, it becomes harder for unauthorized users to gain access or compromise the device.

Implement MFA: Multi-factor authentication (MFA) requires users to provide two or more verification factors to gain access to IoT devices, adding an additional layer of security beyond just username and password. This authentication method is particularly important in IoT environments, where devices often collect or disseminate sensitive data.

Establish regular update processes: Establishing a process for regular firmware and software updates is critical in maintaining IoT security. These updates often contain patches for recently discovered vulnerabilities that could be exploited by attackers. Ensuring that IoT devices are always running the latest version of software helps protect them from such threats and maintains system stability and performance.

Plugging In

The integration of DevSecOps principles into IoT development is useful for addressing the myriad security challenges posed by the diverse landscape of IoT devices. By embedding security practices at every stage of the IoT development lifecycle, from design through deployment, organizations can significantly enhance the security and integrity of their IoT ecosystems.

This approach not only mitigates risks such as unauthorized access and data breaches but also supports the seamless and secure operation of IoT devices. As the IoT continues to evolve, embracing DevSecOps will be crucial for ensuring that innovations in this space are both robust and reliable, safeguarding the digital and physical assets they control.

Photo by Jorge Ramirez on Unsplash

May 20, 2024
Security, Automation and Developer Experience: The Top DevOps Trends of 2024

If you ask most folks to describe the top DevOps trends in 2024, you’ll likely hear buzzwords like AI or DevSecOps.

Those are certainly trendy topics. But based on my everyday work helping businesses plan and execute DevOps strategies, I’m noticing a different set of salient trends in DevOps. Although much is being said about how technologies like AI might impact DevOps, the biggest changes I’m seeing right now involve other types of solutions and techniques.

Here are what I view as the three most important current DevOps trends, along with a breakdown of how they may change DevOps tools and processes.

Policy-Based Management and IaC Drive DevOps Security Innovation

Security has always been a priority for DevOps practitioners. But right now, I’m seeing DevOps teams adopt new strategies to improve the security of the applications and environments they support.

One large-scale change is the greater use of cloud-based policy management to enforce security best practices in cloud environments. Teams configure cloud services and resources using the code-based configuration frameworks that cloud providers support, then scan the configurations to detect risks.

This approach makes it possible to enforce cloud governance consistently, centrally and automatically. Instead of simply writing governance policies and hoping that engineers remember to follow them when they configure cloud resources, businesses are building automated governance guardrails via policy-based management.

In a similar vein, more and more of the DevOps teams I work with are embracing static code analysis of infrastructure-as-code (IaC) templates as a means of detecting risks. As with policy-based management of cloud resources, an IaC-centric approach to infrastructure provisioning makes it possible to automate infrastructure management and identify security risks earlier in the development lifecycle.

Some teams use cloud policy and IaC code scanning to warn one another about security policy changes that might cause an application or service to break. They do this by inserting configuration changes into their code using “audit” mode if their configuration framework supports it, or by configuring IaC scanners to flag changes if an audit feature is not available. This allows engineers to detect whether a change might cause a problem for an existing deployment.

This is important because security and application teams operate separately in many organizations. When the two groups lack an efficient way to communicate about changes, they can disrupt each other’s operations – the “right hand/left hand” problem. Configuration scanning provides a systematic way to ensure that each group is on the same page when it comes time to introduce changes – and the “audit” mode approach provides a grace period that allows time to react before a change takes effect.

Doubling Down on DevOps Automation with GitOps

Another overarching trend currently reshaping DevOps is using GitOps to make DevOps automation more efficient and consistent.

GitOps is the use of Git (or a similar source control system) to manage automated DevOps workflows. It involves defining configurations using code and applying them through features like GitHub actions.

When you opt for GitOps, you move DevOps automation controls from individual workstations to centralized source control repositories. The result is the ability to track and manage all automated workflows via a central hub. Doing so increases efficiency and mitigates issues like different engineers working with different versions of the same automation frameworks on their personal desktops.

In addition, GitOps automatically generates records of what changed and how it changed, because every action is logged through the source control system. This isn’t exactly documentation in the traditional sense and means that GitOps comprehensively documents every change – which is beneficial because human engineers tend not to be so thorough when documenting their actions.

To be sure, GitOps is not without its challenges. Implementing GitOps effectively requires additional skills – namely, expertise with both IaC frameworks and source control systems – that not all DevOps engineers possess. I also notice a tendency on the part of some teams to set up GitOps pipelines but rely on manual approvals instead of automated actions to trigger changes. That approach largely undercuts the value of automating pipelines in the first place.

However, these are challenges that teams can solve through education and by fully leaning into GitOps. In addition, techniques like automated testing of GitOps configuration code can help to build teams’ confidence in automations and reduce reliance on manual approvals.

Going forward, expect to see more adoption of GitOps techniques among teams seeking to level up their approach to DevOps automation. Automating individual DevOps processes (such as software testing and deployment) won’t be enough; truly efficient organizations will turn to GitOps to automate their entire DevOps workflows, from end to end.

Investing in Developer Experience

Making software delivery processes more predictable and efficient is merely a step toward DevOps’s ultimate goal, which is to help developers become productive and satisfied with their jobs.

To that end, I’m noticing interest and investment in developer experience right now. This is playing out through two interrelated initiatives.

One initiative is platform engineering. It involves creating DevOps teams that specialize in certain functions, such as network management or security. Those teams are designated to support those functions throughout the organization. This approach reduces cognitive overhead for developers by freeing them from the need to handle types of work outside their main focus. In other words, instead of forcing developers to be DevOps generalists, platform engineering lets different teams focus on doing what they know and enjoy best – leading to greater productivity and higher levels of job satisfaction.

The other major developer experience trend currently playing out is developer self-service. This means developers can obtain the technical solutions they need on demand without a complicated procurement process. In most cases, organizations enable self-service by implementing internal development platforms (sometimes called internal developer portals) (IDPs) which host ready-made infrastructure resources and software environments that developers can deploy on a self-service basis.

There are risks inherent in these trends. They require specialized types of skills. When poorly implemented, platform engineering and IDP solutions can create more problems than they solve. However, when you ensure that your teams have the requisite expertise, and deploy a carefully planned IDP that gives developers access to the resources they need, you’re likely to see a significant reduction in friction within your organization and a boost in developer productivity and happiness.

Real-World Changes, Beyond the Hype Cycle

Admittedly, discussing DevOps trends centered on security, automation and developer experience may not be as exciting as debating whether AI will take away DevOps engineers’ jobs. But if you want to know what’s truly changing in the world of DevOps – as opposed to which conversations are most hype-worthy – these are the places to look.

Security, automation and developer experience are also among the domains of DevOps where there is a great deal of opportunity to innovate – and, indeed, where adopting new tools and techniques will be critical for organizations that don’t want to be left behind as DevOps evolves.

May 17, 2024
Grafana Labs Makes Observability More Accessible

At a GrafanaCon event today, Grafana Labs unveiled an update to its core Grafana visualization platform. The company is offering an Explore Metric module which it says can identify an issue’s root cause — and to do so without anyone having to launch a query against the open source Prometheus platform that is at the core of the Grafana Cloud observability platform.

That’s not the only enhancement. Version 11 of Grafana also makes it easier to edit and share dashboards, in addition to streamlining the number of alerts that a single incident might generate. In addition, Grafana Labs is now making available a curated edition of the OpenTelemetry software dubbed Grafana Alloy that it will extend and support alongside the rest of its open source software portfolio.

Finally, the company is updating Loki log management to make it simpler to streamline the storage of log data.

Grafana Labs CTO Thomas Wilkie said that collectively these updates will make it easier for a wider range of IT professionals, including application developers, to invoke the capabilities of a Grafana Cloud platform based on open source software.

As IT environments become more complex, the need for observability platforms becomes more pronounced. IT teams typically have access to a raft of monitoring tools for tracking pre-defined metrics but their ability to launch queries to surface the root cause of an issue has been limited. As observability platforms continue to evolve, the line between observability and monitoring continues to blur, said Wilkie.

Less clear is to what degree observability platforms might enable IT teams to rationalize existing monitoring tools. In the meantime, the pace at which IT teams are embracing observability varies widely from one organization to another. IT teams need enough expertise to launch relevant queries to get the full return on their investment in an observability platform. In the absence of that expertise, acquiring an observability platform becomes less compelling, in the short term.

In theory, there soon should come a day when machine learning algorithms and other forms of artificial intelligence (AI) make it easier for organizations to identify issues without having to craft queries. But it’s not likely AI will eliminate the need for site reliability engineers (SREs), said Wilkie. Instead, AI will make it simpler to onboard new members to a DevOps team while deploying more applications than ever, he added.

It’s still early days so far as adoption of AI and observability is concerned but there will be an inevitable convergence. The overall goal should be to make it easier to troubleshoot highly distributed application environments that today span cloud, on-premises and edge computing applications. That’s critical because as application environments become more distributed the overall size of most of the IT teams tasked with managing them is not increasing. As such, that pace at which DevOps workflows can be automated will prove crucial.

In the meantime, IT teams should expect it will become more difficult to triage complex IT environments before it hopefully becomes simpler thanks to the rise of observability, automation and AI.

Photo credit: tito pixel on Unsplash

April 9, 2024
Modern Application Management Requires Deeper Internet Visibility

Most IT teams regularly employ a range of tools to monitor application performance. But in an era where applications are running everywhere from the cloud to the network edge, the visibility being provided is limited. The metrics and analysis being surfaced are roughly akin to only being able to see a few yards ahead when driving a vehicle on a rainy night. Managing modern applications instead requires platforms that provide the equivalent of radar and sonar capabilities, so that IT teams have better visibility into application environments that today remain far too shrouded to effectively manage.

Achieving that goal requires a deeper insight into how various Internet services impact application performance. Just about every application depends on the quality of one or more of those services. The more application services that depend on the Internet, the more probable it becomes that someone will encounter issues intermittently impacting performance in ways that are difficult to understand, much less predict.

Unfortunately, most IT teams today have limited ability to discern how the performance of Internet services is impacting their applications. There are, of course, Internet performance monitoring (IPM) tools capable of surfacing network performance metrics. The challenge and opportunity now is to surface those metrics in context with all the other telemetry data that DevOps teams collect from the various application performance management (APM) and observability platforms they rely on to monitor and troubleshoot application environments.

Why Applications Are Becoming More Distributed

First, with the rise of cloud computing followed by the proliferation of software-as-a-service (SaaS) applications and now the rise of edge computing, applications are becoming more distributed with each passing day. The challenge is that each application is invoked across a network that adds latency, and existing APM and observability platforms cannot effectively analyze it.

The pace at which applications are becoming more distributed is only going to increase. More data is being processed and analyzed at the network edge as organizations deploy near real-time applications. Instead of transferring raw data to the cloud, these applications process and analyze data at the point where it is created and consumed. It’s only then that an application either streams or passes the aggregated results to a cloud service or to an on-premises IT environment for further analysis.

Most DevOps teams, however, lack visibility into how Internet services impact more highly distributed applications, beyond the cloud service or on-premises IT environment where they historically deployed their applications.

Internet Blind Spots

Broadly, there are three major classes of blind spots that impact distributed application performance. The first and arguably most opaque are the services provided by third-party vendors. Ranging from content delivery networks (CDNs) to software-as-a-service (SaaS) application, each of these services is controlled by an external service provider that typically doesn’t allows a DevOps team to collect telemetry data by deploying agent software in their IT environments. At best, they may expose an application programming interface (API) to enable an agentless approach to collecting data, but that method doesn’t typically provide the level of control required to optimize application performance.

The second major blind spot is at the device level. Configuration errors in browsers can have a significant impact on loading times that adversely impacts application performance, for example.

Finally, various platforms and protocols ranging Domain Name Servers (DNS) to publish-and-subscribe services can impact applications in way that an APM or observability platform isn’t going to detect.

The only way to provide the level of visibility required is to leverage an IPM platform that collect telemetry data from a global network, made up of thousands of points-of-presence (PoPs) to determine precisely what level of Internet service is delivered to any given end point.

Context and Control

Integrating the telemetry data collected via an IPM provided DevOps teams with a context to better troubleshoot distributed applications. Instead of wondering if a certain piece of code is to blame, a DevOps team can immediately determine whether Internet traffic moving through a specific location is either creating additional latency or blocking access altogether.

Armed with those insights, the team can contact then Internet services provider to address the issue or, if necessary, reroute via another service to ensure application availability.

Regardless of approach, the days when DevOps teams found themselves subject to the vagaries of network services, over which they had little to no influence, are coming to an end.

DevOps Meets Netops

The integration of IPM and APM or observability platforms creates a foundation upon which best practices for managing network operations (NetOps) and DevOps can finally be melded. Instead of having to convene “war rooms” to uncover the root cause of an issue, both teams will instantly be able to visualize to what degree networking services might be adversely impacting networking services. All that time spent guessing what the cause of a problem might be can now be allocated to remediating the issue.

In an era where applications have never been more dependent on networking services, the need to converge the management of NetOps and DevOps workflows has never been more critical.

Conclusion

Most DevOps teams know all too well there are often major discrepancies between what DevOps teams relying on APM and observability platforms believe and what is actually occurring in a distributed computing environment. The challenge is that they typically lacked the visibility required to determine what best course of action to address the issue. As IPMs become more integrated with APM and observability platforms, network blind spots that previously made discovering the root cause of an issue will soon be all but eliminated.

Of course, each DevOps team needs to determine to what degree gaining those insights is a priority. However, given the number of issues DevOps teams are trying to resolve that can be traced back to an Internet service, it’s clear the overall level of stress and toil DevOps teams regularly encounter could be greatly reduced by simply providing more networking visibility into those services.

April 5, 2024
Bringing DevOps Automation to Legacy Systems

A critical step toward adopting automation for legacy applications is to change your organization’s mindset about legacy systems.

Applying automation to operations isn’t always straightforward, especially with older systems that don’t easily fit into typical DevOps automation tools and strategies. However, legacy systems don’t have to sit on the sidelines of your DevOps automation strategy. Using the right tools and methodologies, it’s possible to integrate legacy system requirements with modern automation techniques.

Most organizations that adopt DevOps for conventional applications use automation tools and techniques, including Infrastructure-as-Code (IaC) and Configuration-as-Code (CaC), automated testing, release automation and application orchestration. If you’ve been paying attention to DevOps for any length of time, you probably are familiar with the concepts.

However, those DevOps automation solutions aren’t always straightforward to implement with legacy systems made up of hardware or software deployed long before modern, DevOps-friendly application architectures and deployment strategies existed.

For example, IaC tools may not support the configuration languages or management tools that the systems require. Most current systems are configured using languages like YAML, but legacy systems may depend on proprietary configuration languages and frameworks.

For similar reasons, test automation can be cumbersome. Today’s automation frameworks are designed for browser-based or mobile apps. Legacy systems may run locally and often use proprietary user interfaces, making automating testing using standard frameworks more difficult.

Likewise, many release automation and application orchestration platforms are designed for applications deployed using containers. Because containers didn’t become popular until the release of Docker in 2013, most legacy systems have limited native support for application containerization.

Bringing Automation to Legacy Systems

Fortunately, these obstacles are possible to overcome. Addressing these challenges requires adopting innovative practices that enable the integration of automation with legacy systems:
• Migrate to the cloud: Legacy system vendors now offer cloud-based hosting platforms. While transferring legacy workloads from on-premises to the Cloud doesn’t automatically enable all DevOps automation features, it does significantly facilitate the use of automatically scalable infrastructure. Organizations can move workloads between cloud data centers to mitigate failures in a single location.
• Containerize legacy workloads: In certain situations, it’s possible to get a legacy application to run in a container. That allows businesses to manage the older software using orchestration solutions like Kubernetes, bringing much greater agility and scalability to legacy application management.
• Use machine learning: Incorporating machine learning into monitoring and management operations for legacy systems helps to detect issues effectively in contexts where conventional automation is not feasible. For example, if you can’t run automated tests for a legacy application because Selenium won’t support it, advanced ML techniques might help you identify bugs based on how the application behaves.
• Change your mindset: Instead of classifying legacy systems as outdated and inefficient environments, treat them with equal respect to all the other applications within your IT estate. When your team approaches legacy systems from a perspective that values them, it becomes much easier to refine a corporate culture toward collaboration and innovation that extends to those legacy systems as well.

To Optimize DevOps Automation, Don’t Ignore Legacy Environments

There’s no denying that automating the management of legacy apps for DevOps teams is often more challenging than implementing automation for modern apps. However, this doesn’t mean that it’s impossible. Many organizations successfully adopt automation as a strategy for improving the flexibility and quality of legacy apps, while also reducing the costs of building and deploying them. But to achieve this, it’s necessary to think beyond the conventional approaches of DevOps.

April 3, 2024
Time to Extend DevOps Visibility to the Network Edge

Performance issues regularly confound DevOps teams after an application is deployed, despite the number of tests run before deployment. Upon further investigation, the issue that is most frequently overlooked is the distributed nature of the application itself. End users accessing applications from multiple locations are never going to have the same level of internet service, so software that may be running perfectly well in locations that are major urban technology hubs, such as New York, San Francisco or London, is not necessarily going to provide the same experience in farther-flung regions, especially rural areas that are halfway around the world.

All internet services are not created equal. Intermittent network latency issues can have a profound impact on application performance simply because of the laws of physics. The farther away an end user is from the data center where the application is hosted, the more likely it becomes network latency will adversely affect application performance.

Unfortunately, most DevOps teams lack any visibility beyond the data center environment where the application is running. Naturally, the first step toward gaining that level of insight starts at the edge of the network.

Internet Stack Visibility

There is a wide range of services that make up an internet stack, each of which can adversely impact the performance of distributed applications. Everything from virtual private networks (VPNs) and content delivery networks (CDNs) to domain name system (DNS) servers and switches adds latency that, depending on how network traffic is being routed, can have a major impact on application performance. Any one of these services might be experiencing an outage or, more challenging to detect, intermittent degradations that unpredictably impact application performance. Whether it’s a software-as-a-service (SaaS) application being accessed in the cloud or an internet-of-things (IoT) platform running at the network edge, pinpointing which internet service might be the root cause of an issue requires the ability to closely monitor and observe that traffic.

Without any visibility into the internet stack, surfacing the insights needed in a way that provides the context required to optimize and troubleshoot a specific application becomes an impossible challenge. All the time and effort spent building an application can come to naught simply because no one understood the impact network latency would have on the application before it was deployed. Even after an application is deployed, any change to the way network traffic is being routed can have major consequences. Without any means of understanding what changes may have occurred, it might be weeks or months before an organization realizes it may need to log a service request or switch service providers.

Any truly comprehensive approach to observability clearly needs to include an analysis of the internet services upon which organizations crucially depend to ensure a consistent application experience is delivered and maintained. Depending on how the IT team is structured, those insights can be fed into an observability platform that the organization has adopted or correlated using an internet performance monitoring (IPM) platform.

DevOps Needs NetOps

Regardless of the approach, it’s never been more crucial for network operations (NetOps) and DevOps teams to be able to align their approaches to monitoring distributed computing environments. More latency-sensitive applications are being deployed than ever. In an ideal world, DevOps teams working closely with NetOps should be able to programmatically provision and alter networking services as needed. Not every internet service can be adjusted via an application programming interface (API), but most NetOps teams have access to consoles that enable them to upgrade to different service levels when needed. At the very least, a service provider will work with them to make any changes required.

Of course, it becomes more challenging to negotiate if the NetOps team doesn’t know which element of the Internet stack needs to be upgraded. A service provider that typically doesn’t have any visibility into the application might spend weeks trying to determine which element of the Internet stack they manage is at fault. IT teams that have their own means of monitoring a set of internet services are going to be able to work with their service providers to resolve those issues much faster.

Summary

Savvy IT teams are all too aware of how prone internet services are to outages and brownouts, so as a rule, they tend to make sure network traffic can be rerouted from one service to another. That approach also provides the added benefit of maintaining some pricing leverage as they negotiate fees for the services provided.

It’s also worth remembering providers of networking services that know a customer is locked in may not have as much incentive to make sure networking services are running at peak efficiency. The only way an IT team is ever going to know what level of service is being delivered at any given time is, of course, to generate an IPM report that precisely details what services were actually available at what level of performance. Otherwise, they are wholly dependent on the report a service provider presents alongside their monthly bill.

Just as critically, the reports generated by an IPM platform also surface insights that, when shown to application developers, create a potential opportunity to optimize their code. Without that level of analysis, every developer will assume the network services are either broken or poorly managed. The challenge and the opportunity now is to provide internet performance intelligence in a way that every stakeholder cannot just act on but, just as critically, implicitly trust.

March 22, 2024
Edge Computing Requires DevOps at Scale

The rise of edge computing is about to drive a long-overdue convergence of DevOps, data engineering, security, networking, operational technology (OT) and machine learning operations (MLOps) best practices, which will ultimately make IT teams more responsive than ever to the needs of the business.

Historically, each of these IT disciplines has tended to work in isolation, with deliberate handoffs being made between the various teams employed to provision and maintain various services. The challenge is that as more applications are deployed at the network edge, the need for IT teams to work hand-in-glove is crucial. Applications not only need to be deployed across a much more distributed computing environment, but they also need to be regularly updated and patched.

At the same time, increasing amounts of data are being processed and analyzed at the point where it is collected and consumed. More stateful applications running at the network edge are passing aggregated analytics data back to applications running in the cloud or on-premises IT environment. That shift requires a fundamentally different approach to how storage is managed across multiple edge computing environments and the various federated backend systems spanning the enterprise.

Organizations will typically need to store files locally on edge computing platforms while also storing copies of those files in the cloud using an S3-compatible storage service that provides access to inexpensive object-based storage to enable everything from data protection to driving artificial intelligence (AI) applications. That multi-protocol capability creates an opportunity to not just build new stateful distributed applications but also modernize existing applications regardless of what protocols are being used to store data.

Most edge computing platforms today are managed by OT teams that report directly to the leaders of specific business units. However, as more of these platforms are connected to networks, the greater the imperative to centrally manage them as part of any effort to lower the total cost of IT. After all, the most expensive element of IT remains the cost of the labor still required to manage it.

Naturally, sharing data in near real-time across a distributed computing environment puts more pressure on network operations (NetOps) teams. Instead of updating applications in batch mode once a day, data continuously streaming from thousands of edge computing platforms needs to be synchronized with multiple applications to surface the latest, most accurate insights.

More challenging still is that much of that data is being used to train AI models that ultimately require the associated inference engine created to be deployed at the edge. To process terabytes of data, the underlying platform required to support those inference engines also will need to be measured in terabytes. Over time, AI models tend to drift or, in the case of generative AI, outright hallucinate, over time. Data scientists will need to work closely with DevOps teams to replace those AI models because, unlike a traditional application, they can’t be updated via a patch. The entire AI model needs to be replaced with one that is not just more reliable but also—and just as importantly—safer.

Finally, the need to secure edge computing platform is nothing short of critical. Cybercriminals tend to view these platforms as gateways to the rest of the enterprise. Each new edge computing platform deployed extends the size of the attack surface that needs to be defended. Unless cybersecurity teams are deeply involved in building and deploying edge computing applications and platforms, it’s only a question of when, rather than if, a breach will occur.

It may be a while before these various IT fiefdoms converge, but at this point, it is all but inevitable. In fact, it will soon be difficult to distinguish between edge computing and the rest of the IT environment as event-driven applications spanning multiple platforms become more the norm than the exception.

DevOps teams, as part of their relentless commitment to automation, should naturally be at the forefront of these efforts. Rather than confining DevOps principles to how applications are built and deployed, the time has clearly come to automate IT workflows on an end-to-end basis. The only way to programmatically manage edge computing at scale is to also apply DevOps principles at scale. Also known as platform engineering, that approach enables organizations to manage distributed IT environments at scale without requiring organizations to hire a small army of IT teams to manage it all. There will always be a need for specialized expertise, but as IT continues to evolve the siloes that slow down the pace of innovation will need to come down.

Not every member of an IT organization needs to know how to expertly manage every task, but they should be familiar with the inherent dependencies that exist between applications, networking, storage and cybersecurity to advance the goals of businesses that have never been more dependent on IT.

Each organization needs to determine the pace of convergence that best suits its needs, but the faster it occurs, the greater the return on the investment in edge computing will become. The challenge and the opportunity, as always, will be managing yet another major IT transition, where the issues that need to be resolved are as much cultural as they are technical.

March 22, 2024
Controlling Cloud Costs With Efficient Ramp Plans

A mix of rapid migrations to the cloud and complexity over usage and costs has companies desperately trying to reduce waste and gain control over their spending. Making matters worse is a volatile economy that has decision-makers on edge, considering cost-cutting measures and placing a priority on locking down revenue.

Without a formal FinOps or cost optimization strategy in place, it’s easy for companies to see cloud expenses snowball out of control. In fact, analysts from Gartner report that organizations that do not employ cloud cost optimization can overspend by up to 70%. That said, ramp plans are becoming a vital tool for containing cloud resources.

Still, while this strategy can show how to effectively grow your environment and increase workloads, the way a ramp plan is carried out is key to its success. The following looks at what companies should do to succeed with ramp planning accompanied by some strategic advice for handling unique needs that may arise.

Ramping Up

A ramp plan, at its most rudimentary level, is data tracking toward how your organization consumes cloud resources versus what it commits to with a cloud service provider (CSP) or reseller. By committing to spend a set dollar amount, you can get healthy discounts or other incentives. As an example, a 50% increase in consumption by the end of the year might earn a 10% discount from a CSP on compute workloads. But there is risk: A company is still obligated to pay that figure, even if it uses far fewer resources than it committed to.

Still, just as you don’t want to spend money needlessly, CSPs—even high-volume hyperscalers like Amazon AWS and Google GCP—would rather see customers get what they need and grow their footprint. For some organizations, the answer is bringing in a partner to help forecast spend. Questions regarding what is needed, past usage, anticipated growth, potential for acquisitions and more need to be answered. You also have to examine such things as product and service offerings. For instance, a subscription app you offer could catch fire and bring in customers—but you’ll need to plan accordingly so the resources will be in place to meet this demand.

Compiling and projecting this all isn’t easy, but this type of data is needed so you can develop a ramp plan and budget effectively. In the past, this process required time-consuming manual research, a host of spreadsheets and sophisticated algorithms. However, technology has been developed to automate, drive and scale this process. For some businesses, the ideal is to find a partner with this capability in-house and a depth of expertise to use it correctly.

Processes and Partners

Ideally, a ramp plan should be developed before a commitment is made. It should be reviewed on at least a quarterly basis with the C-suite and appropriate technical executives and then revised as necessary. On the whole, CSPs like to sign customers to contracts extending more than a year, with three-year terms being typical.

Working with a third-party partner, reseller or directly with a hyperscaler comes down to personal preference. Big companies, much like hyperscalers, tend to have a lot of bureaucratic processes and requirements. So, if you want a tactical agreement with AWS, you may have to buy tactical support. You may get a good financial incentive in exchange, but the cost of enterprise-level support could offset that.

Partners and resellers of cloud services, on the other hand, help hyperscalers scale their businesses, so they often have special arrangements that lower support costs. The development of ramp plans is usually less when going through a third party versus registering with a CSP. These services are often designed for smaller companies that larger CSPs don’t have time to handle. Third parties tend to provide tailored services more in tune with customer needs, including more flexible usage, optimization and security. These partners tend to negotiate more effectively than customers themselves because they better understand the ecosystem and which levers to pull.

Measuring Success

It’s not easy to measure and track the success of a ramp plan. Beyond understanding and meeting your commitment—and getting the most from cloud spend—the best measure is profitability. Are you just eating costs or acquiring new customers? Is your efficiency boosting or lowering your margins? It’s even possible to apply unit economics to assess if you’re making money on the resources being used.

Making a commitment and ramp planning should involve more than financial incentives. Maybe there’s some other strategic aspect you want to achieve? From service-level agreements to support, a lot of things can be woven into a commercial contract. Benefits can come in various forms, but when they’re added together, you can take success in the cloud to a whole new level.

March 14, 2024
The IaC Weight on DevOps’ Shoulders
It’s often challenging for established tech companies to adopt modern DevOps practices around infrastructure-as-code (IaC). It’s not so much in adopting the syntax, but rather the DevOps culture that IaC is part of for engineering organizations.

Organizations often struggle to transition from traditional operations models to collaborative DevOps culture, which is a byproduct of transitioning infrastructure operations to code.

In many cases, legacy tendencies persist, while IaC duties stay siloed within ops-centric DevOps teams—which is the exact opposite of the benefits of leveraging powerful technology like infrastructure-as-code. What this ultimately creates is bottlenecks in development velocity and product iteration.

With all this in mind, it begs the question: What changes can enable a smoother IaC journey?

The Pitfalls of IaC Silos

Having DevOps teams shoulder the entire IaC burden is the exact anti-pattern we’re looking to avoid when truly transitioning to DevOps culture in our organizations. What many organizations start to experience is these familiar pain points that start to emerge:

1.) Knowledge Gaps Widen

Developers—by nature of their day job as software engineers—often lack access, visibility and context into infrastructure changes. Yet, with more and more infrastructure requirements shifting left, they are left (pun intended) with insufficient capabilities to manage these growing needs.

This is compounded when they are faced with minimal cloud permissions. Limited permissions make it difficult to experiment and familiarize themselves with cloud tools and capabilities the way they would with code and its parts of the stack.

Meanwhile, DevOps teams—as the exact mirror image and the nature of their role in the organization—don’t grasp application needs and objectives. This often results in a widening chasm of broken communication and collaboration around IaC initiatives.

2.) Collaboration Suffers

A common tool that exacerbates the problem is a ticketing and support platform. Developers, as their modus operandi, lacking a shared understanding of the domain, will often resort to opening tickets to achieve their goals. This creates a culture of inefficient cross-team communication via ticketing systems.

This doesn’t mean the ticketing platforms are the problem, it just results in inefficient communication methods, as developers don’t always know what questions to ask.

What this does mean is that DevOps engineers operate largely on instinct, being mostly in the dark on the real application requirements that can’t all be communicated in a simple Jira ticket. This results in final solutions that rarely satisfy needs on the first attempt and take longer to deliver.

3.) Releases Slow to a Crawl

DevOps teams hold the responsibility for conducting and orchestrating release engineering, on top of provisioning environments, maintenance, monitoring and other tasks that fall in the DevOps engineering domain.

This is why DevOps tasks can become a bottleneck, as they not only dictate deployment timing (due to important considerations of uptime and client disruption, among others) but are also tasked with juggling infrastructure changes across many and multiple environments.

This causes unpredictable delays in provisioning dev environments and other developer-oriented tasks that frustrate developers and stall production updates, and delivery velocity.

The Obstacles of Change

Recognizing these inefficiencies and shifting these responsibilities left became the popular way for DevOps leaders to distribute IaC duties. This allowed developers to gain sufficient autonomy to choose the solutions for their needs, while DevOps teams provided guidance and oversight for how to practically achieve these solutions.

However, like all things shift left, this comes with its own set of unique challenges. Adopting the DevOps paradigm faces fundamental obstacles that add further complications and complexity to these cultural transitions, including:
- Uncontrolled sprawl
- Ballooning costs
- Security concerns
On the one hand, distributing the IaC load lessens the burden on the DevOps teams, but the downside is that it becomes difficult to understand which resources are actually in use and which have been temporarily created for testing purposes.

With many owners creating resources on demand, once they are no longer needed, these leftovers create confusion around dependencies and make cloud platforms disorganized and difficult to maintain.

Just like enabling more hands to touch IaC creates greater sprawl and disorder, more users with less governance invite careless sprawl in terms of costs as well. This often results in duplicate and unused resources accumulating, wasting budgets that are currently tight, and every penny counts. With a lack of automation and oversight, environments grow messy and expensive.

The sprawl issues can also impact security, as expanding permissions raises valid security concerns that are intensified when clouds become disorganized and difficult to maintain.

Well-intentioned developers may misconfigure resources or expose sensitive systems, and without proper methods to manage drift or misconfiguration, this can pose real risks to organizations and systems. Another important aspect that also increases with less oversight is intentional insider risk.

From Silos to Shared Responsibility

OK, that must have been really disheartening—and you’re probably thinking you should run for the hills before attempting any transitions to shared responsibility.

But wait—don’t panic yet. It is possible for teams to redistribute IaC responsibility while avoiding these pitfalls.

Below are some key factors to help you make the transition smoother in the short and the long term:
- Automate everything—If DevOps practices have taught us anything, it is that manual cloud environment maintenance doesn’t scale. Automation has been the backbone of much of the evolution we have undergone when it comes to infrastructure operations, and this has been powered by the programmability and everything-as-code mindset—from IaC to policy-as-code (PaC). This automation-based optimization makes it possible to reduce neglected resources through regular and automated analysis and response. What you get is cost savings and the prevention of runaway sprawl.
- Empower through governance—Remember when we spoke about automation and governance? Well, programmatic policy enforcement is a good way to constrain unwanted resource proliferation without manual intervention. Governance comes in to handle auto-remediation of misconfigurations or other anti-patterns that do not align with automated policies, which simplifies environment management at scale. Importantly, governance, if done right, can be a tool to empower teams, break silos and encourage autonomy. Having the right guardrails in place is how you instill confidence and provide helpful direction.
- RBAC for the win—Role-based access control (RBAC) limits permissions to appropriate needs, establishing important guardrails that can help prevent some of the issues we spoke about above, including sprawl and security concerns.
RBAC will enable developers to gain the necessary freedom while minimizing security risk. Minimum required access encourages responsibility and constraint.

Now that you have a solid grasp of the principles, it’s time to put them into action!

Fortunately, there are numerous solutions at your disposal. You can jump in and build them yourself using capable tools like Jenkins or Atlantis.

Alternatively, for a more comprehensive approach, there are excellent commercial IaC management tools on the market. These can assist you in reaching your objectives swiftly and with greater efficiency.

An Evolutionary Vision for IaC and DevOps

As technology complexity increases, no single team can handle infrastructure alone. Dev and Ops must unite abilities to shepherd modern systems and the underlying code powering them. This requires closing knowledge gaps through collaboration and communication

With a thoughtful process, IaC automation, governance guardrails and the right access policies, tech leaders can guide this cultural transition smoothly without exposing the organization to unnecessary risk and spending.

When done right, Shared IaC responsibility will boost velocity, agility, and autonomy—the trifecta of crucial traits that quantify and qualify elite engineering teams and differentiate companies in a competitive landscape.

Interested in learning more about env0 to manage your IaC at scale and confidently?

Stop by Booth L12 at KubeCon EU to speak with an expert or see a demo!

To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.
March 13, 2024