devopsy.kinsta.cloud

Tag: networking

Overcoming IP Churn in Ephemeral DevOps Environments Using Userspace Overlays

Modern DevOps practices have completely transformed how we handle compute and orchestration. Tools like Kubernetes enable engineering teams to spin up ephemeral containers in seconds and scale workloads dynamically to meet global demand. Yet the underlying network infrastructure has remained stubbornly rigid. Traditional cloud networking relies heavily on static IP addresses, rigid firewall rules, and slow DNS propagation. This creates a severe architectural mismatch where highly dynamic compute layers are trapped inside static network topologies.

This friction becomes a critical failure point when deploying modern distributed workloads. Artificial intelligence applications and autonomous multi-agent systems are inherently transient. They migrate across cloud providers to optimize for GPU availability or drop down to local edge devices for data collection. Every time a container restarts or shifts to a new environment, its physical IP address changes. This IP churn breaks stateful connections and forces DevOps engineers to build fragile workarounds.

To keep systems running, the DevOps community has engineered a variety of solutions to abstract the network. Teams heavily invested in Kubernetes often deploy service meshes like Istio or Linkerd to manage internal cluster traffic. For cross-environment service discovery, HashiCorp Consul provides a robust central registry. When bridging physical networks, overlay VPNs like Tailscale and ZeroTier create secure meshes, while eBPF solutions like Cilium push routing logic deep into the Linux kernel. Recently, engineering teams have started adopting pure userspace overlay networks like Pilot Protocol to bypass kernel constraints entirely and embed peer-to-peer routing directly into the application layer.

Each of these solutions offers distinct advantages, but they also introduce specific operational tradeoffs. Service meshes and centralized registries require significant infrastructure overhead and dedicated control plane management. Kernel-level overlays deliver incredible performance but demand root privileges and host-level network interface modifications. This creates massive friction in heavily restricted serverless environments or unprivileged continuous integration pipelines where developers cannot modify the underlying host operating system.

The root of the problem is that legacy internet protocols bind software identity to physical hardware locations. If a node moves to a new server, it loses its identity. When bridging isolated virtual private clouds, cloud providers typically force teams into configuring complex Transit Gateways or IPSec tunnels. These topologies demand meticulous IP address management to prevent subnet overlap and require constant manual routing table updates.

Workarounds like Dynamic DNS attempt to automate this process by rapidly updating records when a container shifts locations. However, DNS was never designed for real-time tracking of ephemeral microservices. Aggressive caching by internet service providers and propagation delays mean that distributed workloads lose connectivity for minutes at a time during every migration. In a high-velocity DevOps environment, relying on DNS for stateful service discovery introduces unacceptable deployment fragility.

To build truly resilient continuous deployment pipelines, DevOps teams must decouple network identity from physical infrastructure. The emerging solution to this bottleneck is shifting network routing entirely into userspace via cryptographic overlay networks.

A userspace overlay network operates above the operating system kernel and abstracts the physical network away entirely. Instead of relying on a cloud provider to assign a temporary IP address, the software node generates a permanent cryptographic keypair upon initialization. The public key serves as the node’s immutable network address. Whether the application is running in an AWS pod, an Azure serverless function, or a local developer laptop, its address remains mathematically identical on the overlay network.

This approach drastically reduces the operational burden on network engineering teams. Userspace overlays bypass infrastructure friction entirely by utilizing automated UDP hole-punching. When two distributed nodes need to communicate, they leverage the overlay to negotiate a direct peer-to-peer connection through their respective Network Address Translation firewalls. The traffic is routed securely without requiring manual port forwarding or centralized transit hubs.

Implementing this architecture enforces a strict DevSecOps posture by default. Because the network address is derived directly from a cryptographic key, all connections are inherently authenticated. Before any application data is exchanged, nodes perform a zero-trust handshake utilizing Elliptic Curve Diffie-Hellman key exchange and AES authenticated encryption.

Malicious actors cannot spoof physical IP addresses to intercept traffic because the network validates the cryptographic signature of the endpoint. Furthermore, listening application sockets are never exposed to the public internet. This drastically reduces the external attack surface and simplifies compliance audits. Security teams no longer need to manage thousands of complex access control lists because the network boundary itself acts as the primary trust boundary.

The future of DevOps depends on removing the friction between application deployment and network configuration. Forcing highly mobile, autonomous software to navigate the rigid constraints of traditional IP architecture creates administrative debt and brittle delivery pipelines. By adopting identity-based userspace overlays, engineering teams can further abstract the physical network from the applications running on it. This enables developers to deploy secure, globally distributed systems with the same velocity and automation they apply to their application code.

June 1, 2026
Time to Extend DevOps Visibility to the Network Edge

Performance issues regularly confound DevOps teams after an application is deployed, despite the number of tests run before deployment. Upon further investigation, the issue that is most frequently overlooked is the distributed nature of the application itself. End users accessing applications from multiple locations are never going to have the same level of internet service, so software that may be running perfectly well in locations that are major urban technology hubs, such as New York, San Francisco or London, is not necessarily going to provide the same experience in farther-flung regions, especially rural areas that are halfway around the world.

All internet services are not created equal. Intermittent network latency issues can have a profound impact on application performance simply because of the laws of physics. The farther away an end user is from the data center where the application is hosted, the more likely it becomes network latency will adversely affect application performance.

Unfortunately, most DevOps teams lack any visibility beyond the data center environment where the application is running. Naturally, the first step toward gaining that level of insight starts at the edge of the network.

Internet Stack Visibility

There is a wide range of services that make up an internet stack, each of which can adversely impact the performance of distributed applications. Everything from virtual private networks (VPNs) and content delivery networks (CDNs) to domain name system (DNS) servers and switches adds latency that, depending on how network traffic is being routed, can have a major impact on application performance. Any one of these services might be experiencing an outage or, more challenging to detect, intermittent degradations that unpredictably impact application performance. Whether it’s a software-as-a-service (SaaS) application being accessed in the cloud or an internet-of-things (IoT) platform running at the network edge, pinpointing which internet service might be the root cause of an issue requires the ability to closely monitor and observe that traffic.

Without any visibility into the internet stack, surfacing the insights needed in a way that provides the context required to optimize and troubleshoot a specific application becomes an impossible challenge. All the time and effort spent building an application can come to naught simply because no one understood the impact network latency would have on the application before it was deployed. Even after an application is deployed, any change to the way network traffic is being routed can have major consequences. Without any means of understanding what changes may have occurred, it might be weeks or months before an organization realizes it may need to log a service request or switch service providers.

Any truly comprehensive approach to observability clearly needs to include an analysis of the internet services upon which organizations crucially depend to ensure a consistent application experience is delivered and maintained. Depending on how the IT team is structured, those insights can be fed into an observability platform that the organization has adopted or correlated using an internet performance monitoring (IPM) platform.

DevOps Needs NetOps

Regardless of the approach, it’s never been more crucial for network operations (NetOps) and DevOps teams to be able to align their approaches to monitoring distributed computing environments. More latency-sensitive applications are being deployed than ever. In an ideal world, DevOps teams working closely with NetOps should be able to programmatically provision and alter networking services as needed. Not every internet service can be adjusted via an application programming interface (API), but most NetOps teams have access to consoles that enable them to upgrade to different service levels when needed. At the very least, a service provider will work with them to make any changes required.

Of course, it becomes more challenging to negotiate if the NetOps team doesn’t know which element of the Internet stack needs to be upgraded. A service provider that typically doesn’t have any visibility into the application might spend weeks trying to determine which element of the Internet stack they manage is at fault. IT teams that have their own means of monitoring a set of internet services are going to be able to work with their service providers to resolve those issues much faster.

Summary

Savvy IT teams are all too aware of how prone internet services are to outages and brownouts, so as a rule, they tend to make sure network traffic can be rerouted from one service to another. That approach also provides the added benefit of maintaining some pricing leverage as they negotiate fees for the services provided.

It’s also worth remembering providers of networking services that know a customer is locked in may not have as much incentive to make sure networking services are running at peak efficiency. The only way an IT team is ever going to know what level of service is being delivered at any given time is, of course, to generate an IPM report that precisely details what services were actually available at what level of performance. Otherwise, they are wholly dependent on the report a service provider presents alongside their monthly bill.

Just as critically, the reports generated by an IPM platform also surface insights that, when shown to application developers, create a potential opportunity to optimize their code. Without that level of analysis, every developer will assume the network services are either broken or poorly managed. The challenge and the opportunity now is to provide internet performance intelligence in a way that every stakeholder cannot just act on but, just as critically, implicitly trust.

March 22, 2024
Edge Computing Requires DevOps at Scale

The rise of edge computing is about to drive a long-overdue convergence of DevOps, data engineering, security, networking, operational technology (OT) and machine learning operations (MLOps) best practices, which will ultimately make IT teams more responsive than ever to the needs of the business.

Historically, each of these IT disciplines has tended to work in isolation, with deliberate handoffs being made between the various teams employed to provision and maintain various services. The challenge is that as more applications are deployed at the network edge, the need for IT teams to work hand-in-glove is crucial. Applications not only need to be deployed across a much more distributed computing environment, but they also need to be regularly updated and patched.

At the same time, increasing amounts of data are being processed and analyzed at the point where it is collected and consumed. More stateful applications running at the network edge are passing aggregated analytics data back to applications running in the cloud or on-premises IT environment. That shift requires a fundamentally different approach to how storage is managed across multiple edge computing environments and the various federated backend systems spanning the enterprise.

Organizations will typically need to store files locally on edge computing platforms while also storing copies of those files in the cloud using an S3-compatible storage service that provides access to inexpensive object-based storage to enable everything from data protection to driving artificial intelligence (AI) applications. That multi-protocol capability creates an opportunity to not just build new stateful distributed applications but also modernize existing applications regardless of what protocols are being used to store data.

Most edge computing platforms today are managed by OT teams that report directly to the leaders of specific business units. However, as more of these platforms are connected to networks, the greater the imperative to centrally manage them as part of any effort to lower the total cost of IT. After all, the most expensive element of IT remains the cost of the labor still required to manage it.

Naturally, sharing data in near real-time across a distributed computing environment puts more pressure on network operations (NetOps) teams. Instead of updating applications in batch mode once a day, data continuously streaming from thousands of edge computing platforms needs to be synchronized with multiple applications to surface the latest, most accurate insights.

More challenging still is that much of that data is being used to train AI models that ultimately require the associated inference engine created to be deployed at the edge. To process terabytes of data, the underlying platform required to support those inference engines also will need to be measured in terabytes. Over time, AI models tend to drift or, in the case of generative AI, outright hallucinate, over time. Data scientists will need to work closely with DevOps teams to replace those AI models because, unlike a traditional application, they can’t be updated via a patch. The entire AI model needs to be replaced with one that is not just more reliable but also—and just as importantly—safer.

Finally, the need to secure edge computing platform is nothing short of critical. Cybercriminals tend to view these platforms as gateways to the rest of the enterprise. Each new edge computing platform deployed extends the size of the attack surface that needs to be defended. Unless cybersecurity teams are deeply involved in building and deploying edge computing applications and platforms, it’s only a question of when, rather than if, a breach will occur.

It may be a while before these various IT fiefdoms converge, but at this point, it is all but inevitable. In fact, it will soon be difficult to distinguish between edge computing and the rest of the IT environment as event-driven applications spanning multiple platforms become more the norm than the exception.

DevOps teams, as part of their relentless commitment to automation, should naturally be at the forefront of these efforts. Rather than confining DevOps principles to how applications are built and deployed, the time has clearly come to automate IT workflows on an end-to-end basis. The only way to programmatically manage edge computing at scale is to also apply DevOps principles at scale. Also known as platform engineering, that approach enables organizations to manage distributed IT environments at scale without requiring organizations to hire a small army of IT teams to manage it all. There will always be a need for specialized expertise, but as IT continues to evolve the siloes that slow down the pace of innovation will need to come down.

Not every member of an IT organization needs to know how to expertly manage every task, but they should be familiar with the inherent dependencies that exist between applications, networking, storage and cybersecurity to advance the goals of businesses that have never been more dependent on IT.

Each organization needs to determine the pace of convergence that best suits its needs, but the faster it occurs, the greater the return on the investment in edge computing will become. The challenge and the opportunity, as always, will be managing yet another major IT transition, where the issues that need to be resolved are as much cultural as they are technical.

March 22, 2024
Cloudflare Unveils Platform to Programmatically Unify Cloud Networking

Cloudflare today launched a Magic Cloud Networking platform that makes it simpler to programmatically manage multiple cloud networking environments using Terraform code.

Based on a platform the Cloudflare gained when it acquired the core technology from Nefeli Networks late last year, the Magic Cloud Networking platform presents IT teams with a console through which they can use the existing Cloudflare One secure access service edge (SASE) platform to securely manage multiple cloud networks. The Cloudflare global network spans more than 310 cities in over 120 countries and interconnects with more than 13,000 networks worldwide.

Annika Garbers, director of product for Cloudflare, said this addition to the Cloudflare portfolio promises to make it simpler to integrate network operations (NetOps) with DevOps workflows in a way that makes it simpler to enforce zero-trust IT policies.

The degree to which NetOps and DevOps are integrated tends to vary widely from one organization to the next. It’s still not uncommon for DevOps teams to be able to provision a virtual machine in seconds or minutes but then be forced to wait days for a NetOps team to provide then network access. Arguably, those kinds of delays defeat the purpose of investing in DevOps in the first place.

That’s becoming a more problematic issue as organizations start to deploy workloads across multiple clouds, each with its own construct for managing software-defined networking infrastructure. The Magic Cloud Networking platform provides DevOps teams with a higher level of abstraction for programmatically managing those networks using Terraform code that is already widely used to manage infrastructure-as-code, noted Garbers.

That approach eliminates the need to rely on cumbersome virtual routers that IT teams don’t want to manage. In fact, more IT teams than ever just want to be able to access network services versus having to build, deploy and manage physical routers and switches, said Garbers. In the age of the cloud, the physical network underlay is typically managed by the cloud service provider, so all that’s required is a way to programmatically unify the management of multiple cloud networks that can be invoked via lower-level application programming interfaces (APIs).

The one thing that is certain is that as networking services become more programmable, the pace at which applications can be deployed is only going to accelerate. Each organization will need to determine which teams are best suited to manage any given layer of the networking stack. Still, regardless of approach it’s clear networks are finally catching up to rest of the IT infrastructure that is already being managed using code. The challenge, as always, is making just as easy for network administrators that lack programming skills to manage those networks as easily as a DevOps team that already has extensive experience using code to manage the rest of the infrastructure used to provide IT services.

Of course, there will still be plenty of routers and switches to manage in on-premises IT environments but it’s only a matter of time before they become part of a larger DevOps workflow.

March 6, 2024
ngrok Previews API Gateway Service to Streamline App Networking

Under an early access program, ngrok today made available an application programming interface (API) gateway that can be consumed as a service.

Rather than having to deploy, maintain, and secure API gateways themselves, application development teams, via a single command or function call, can now invoke a service through which they can route API traffic, authenticate and authorize access, and apply rate-limiting policies using an ngrok policy engine.

ngrok CEO Alan Shreve said this approach also makes it simpler for IT teams to unify ingress management at a time when application networking is becoming more challenging to manage. That approach also reduces friction because instead of being dependent on an IT operations team, it gives developers more direct control over APIs, he added. Today, most access to APIs is granted after developer first create a ticket request in an IT services management (ITSM) platform, noted Shreve.

In addition, the ngrok gateway also streamlines access to APIs spanning multiple clouds and on-premises IT environments in a way that enables organizations to implement a multi-cloud computing strategy, he added. Today, many organizations are locked into a single platform because they are dependent on the API gateway services provided by their cloud service provider, noted Shreve.

Finally, organizations will only need to pay for the API gateway service based on actual usage versus incurring the cost of acquiring a gateway upfront, said Shreve.

Previously, ngrok has been making available software development kits (SDKs) to provide programmable access to its API gateway. The as-a-service option being offered now is the next logical extension of that effort, noted Shreve.

In general, more infrastructure than ever is being consumed as a service, so making available an API gateway service is only the latest example of that larger trend, he added.

Each organization, based on total cost, will naturally need to decide when it makes the most economic sense to consume IT infrastructure as a service; however, as application networking continues to become more challenging to manage at a time when the number of IT professionals with that expertise remains limited. Relying on a service provides a viable alternative for managing what are often hundreds of APIs spanning a distributed computing environment.

It’s not clear yet precisely which teams within an IT organization are assuming responsibility for application networking, but as layer 4 through layer 7 of the networking stack become more programmable, more responsibility for networking can be assumed by DevOps and platform engineering teams. Existing networking teams, meanwhile, continue to manage the physical networking underlays used to route network traffic. Regardless of how application networking evolves, the rigidity that has characterized the delivery of network services is starting to fade away in a way that makes it possible to dynamically invoke networking services as needed.

IT teams must decide how best to approach application networking across legacy monolithic applications, microservices and event-driven applications. The goal, however, should be to eliminate as much friction as possible at a time when applications have become more dependent on networking services than ever.

February 15, 2024
F5 Looks to Streamline Application Networking

At its AppWorld 2024 conference this week, F5 announced a software-as-a-service (SaaS) edition of its NGINX application networking portfolio, which are now all available under a single enterprise license.

Shawn Wormke, general manager for NGINX, said NGINX One will eliminate much of the friction enterprise IT teams currently encounter when implementing NGINX Plus, NGINX Open Source, NGINX Unit, NGINX Gateway Fabric and the Kubernetes Ingress Controller the company provides.

As part of that effort, F5 is also now making available a unified console across its application networking portfolio that promises to streamline the management of the F5 application networking portfolio.

Currently available under an early access program, NGINX One creates an as-a-service platform through which enterprise IT organizations can invoke and pay for these capabilities on an as-needed basis, said Wormke. The overall goal is to reduce the total cost of application networking by eliminating the need for IT teams to integrate the various elements of NGINX One on their own, he added.

In general, application networking has become more complex with the rise of cloud-native applications based on microservices. By reducing the cognitive load required to network those applications, it becomes simpler to make network operations a more natural extension of DevOps workflows, noted Wormke.

Most organizations have some level of experience with application networking using proxy servers, but as application environments have become more complex, the need for gateways and ingress controllers to provide these capabilities at scale has become more apparent. As a result, more IT organizations are starting to revisit how they are structured. There may always be a need for dedicated networking specialists to manage the physical network underlay, but as other networking services become more integrated with DevOps workflows, responsibility for some networking services is starting to shift further left toward DevOps teams that will either deploy these capabilities themselves or take advantage of a SaaS platform such as NGINX One. The overall goal is to be able to dynamically provision application networking services without having to wait for a network administrator to provision them.

There are, of course, a lot of approaches to application networking that are now emerging. Regardless of how application networking evolves, however, the rigidity that has characterized the delivery of network services for decades should finally start to fade away. Today, outside of a cloud computing environment, it’s still common for IT teams to provision virtual machines or Kubernetes pods in hours only to wait days or even weeks for network connectivity to be provisioned. As IT environments become more hybrid and more workloads are pushed to the edge, there is a clear need for a more agile approach.

Each organization will need to decide how best to approach application networking across legacy monolithic applications, microservices and event-driven applications at a time when IT environments have become much more complex to manage. The challenge, as always, is reducing as much friction as possible without unduly raising the total cost of IT to a point where it becomes unsustainable.

February 8, 2024
2024 Infrastructure Tech Predictions

Ganesh Srinivasan, partner at Venrock, co-authored this article.

2023 was a rollercoaster like none other; from the death of the modern data stack sprawl to the birth of generative AI, we are only at the beginning of a new era in the ‘art of the possible.’ We guarantee 2024 won’t be a disappointment.

With a new year approaching, it’s the perfect time for us to examine what we anticipate being the biggest developments in the year ahead. Here is what we think is going to happen in 2024:

1. OpenAI’s Reign Challenged

With the emerging learnings in core neural net architectures that led to the transformer and dominance by OpenAI, it is likely that their imminent release of GPT5 will be surpassed in specific performance benchmarks by a new entrant on the backs of more efficient architectures, improved multimodal capabilities, better contextual understanding of the world and enhanced transfer learning. These new models will be built on emerging research in spatial networks, graph structures and combinations of various neural networks that will lead to more efficient, versatile and powerful capabilities.

2. Apple: The New Leader in Generative AI

One of the most important players in the generative AI space is only starting to show their cards. 2024 will be the year Apple launches its first set of generative AI capabilities, unlocking the true potential of an AI-on-the-edge, closed architecture with full access to your personal data – showing that Apple is actually the most important company in the generative AI race.

3. Building for Client-First

The last decade has reflected a shift away from fat clients to server-side rendering and compute. But the world is changing back to the client. Mobile-first experiences will be required to work in offline mode. Real-time experiences require ultra-low latency transactions. Running LLMs will increasingly be required to run on the device to increase performance and reduce costs.

4. Death of Data Infrastructure Sprawl

The rapid growth of the data infrastructure needs of enterprises has led to an increasing sprawl of point solutions, from data catalogs, data governance, reverse extract, transform, load, and airflow alternatives to vector databases and yet another lakehouse. The pendulum will swing back to unified platforms and fewer silos to bring down the total cost of ownership and operating overhead going into 2024.

5. Approaching the AI Winter

Generative AI in 2023 could be best characterized as the ‘art of the possible,’ with 2024 being the true test to see if prototypes convert into production use cases. With the peak of the hype cycle likely here, 2024 will experience the stage of disillusionment where enterprises discover where generative AI can create margin-positive impact and where the costs outweigh the benefits.

6. The Misinformation Threat

While image and video diffusion models have unlocked a new era for digital creation and artistic expression, there’s no doubt that its dark side has not yet taken its toll. With a presidential election in the wings, diffusion models as a machine for political disinformation will emerge to become the next major disinformation weapon of choice.

7. AI’s Real-World Breakthrough

Coming out of the ‘field of dreams’ era for AI, 2024 will represent a breakthrough for commercial use cases in AI, particularly in the physical world. Using AI for physical world modalities will unlock our ability to change and engage with the machines and devices around us, from understanding machine telemetry for problem detection to distilling insights across thousands of real-time sensor feeds.

8. S3: The Growing Threat to Snowflake

AWS S3 is about to become the biggest competitor to Snowflake. With a rapidly growing ecosystem of tools emerging as the compute layers on top of S3 leverage their ultra-low costs, high availability, Availability Zone support, and new single-digit millisecond latency service, everyone’s ‘Snowflake cost problem’ is about to have a new set of attractive alternatives.

9. The Rise of Flink and Operational Analytics

If business analytics drove the first wave of the data infrastructure evolution, operational analytics will drive the next wave, with Apache Flink at the center of it. Enterprises will begin en masse adopting real-time and streaming capabilities to begin using data to drive operational decision making. This will create new value in areas such as logistics, financial transactions, inventory management and fraud detection.

10. The Developer AI Stack

Going into 2024, AI’s most significant impact in the digital world will be unlocking developer productivity. From code generation to debugging to build management, AI will unlock a new level of developer productivity that we haven’t seen since the advent of the cloud.

If you agree, disagree or would like to share your own 2024 predictions with us, please reach out on X at @ethanjb and @gan3sh.

January 23, 2024
Boost User Experience With Global Network Monitoring
With the increasing global use of apps and services, ensuring a smooth user experience across different geographic locations is already a top priority for many DevOps teams. However, traditional setups, such as maintaining servers in multiple locations, can be a costly and inflexible approach, especially for smaller DevOps teams or those with tight budgets.

In this article, we’ll explore how DevOps teams of all sizes can leverage global network testing platforms to set up simple and flexible global monitoring that fits well with existing workflows without breaking the bank.

Operating on a Global Scale Requires Flexibility

To monitor apps and services from various locations around the globe, teams need access to a globally distributed infrastructure.
Typically, DevOps teams consider different setups to meet their needs, including:
- On-premises servers: Larger companies with data centers or offices worldwide can use these locations to set up network monitoring and troubleshooting.
- Cloud service providers: Infrastructure-as-a-service (IaaS) solutions allow teams to deploy virtual servers to multiple locations without needing physical access to those sites.
- Third-party services: Feature-rich monitoring tools with servers in multiple regions enable teams to leverage existing infrastructure that they don’t have to maintain themselves.
While these setups offer various benefits to different project and team requirements, they also come with drawbacks that can impact a DevOps team’s efforts:
- Cost: Infrastructure, whether self-managed or acquired via third-party vendors, can be a significant expense – especially when teams need to monitor and test from multiple locations. Moreover, third-party monitoring services may come with unneeded features, resulting in unnecessary costs.
- Flexibility: Most setups don’t allow users to change the location from which to test or monitor as needed, making it impossible to work on a real global scale. Plus, they can often not cover all the areas from which teams want to make measurements.
- Complexity: Adding new hardware, cloud instances and tools to workflows introduces new complexity and challenges, especially for DevOps teams that already struggle to complete tasks efficiently.
To address these issues, teams should consider deploying a global network testing platform that allows them to run network tests globally and overcome the challenges described.

Leveraging Global Network Testing Platforms

Global network testing platforms such as Globalping or RIPE Atlas provide a reliable solution for effortlessly performing network measurements from any location worldwide. Note, however, that these platforms are quite different from extensive continuous monitoring platforms like Kentik. They are much more streamlined and focus on delivering on-demand network testing capabilities through APIs, CLIs and other integrations.

But how do these platforms enable teams to run network tests from anywhere? They do this by leveraging a dense network of thousands of probes hosted by partners and individuals worldwide. These probes serve as the hosts from which network commands, such as ping, traceroute, mtr or dig, are executed. As a result, platform users can test and examine various aspects of their website, CDN or DNS server – even in the far corners of the world.

This image shows an example of the Globalping CLI displaying ping results to devopsy.kinsta.cloud from probes in Berlin and South America:

Transform On-Demand Testing Into Monitoring

While on-demand testing and monitoring are not exactly the same, global network testing platforms can still serve most DevOps teams’ monitoring setups.

These platforms usually provide APIs and CLI tools that let users make measurement requests programmatically so they can use these integrations for their own purposes.

Here are two practical and straightforward use cases to turn on-demand testing into monitoring:
- Teams can create customized global monitoring tools using the platform’s API or extend their existing setups. They can make measurement requests from various locations, collect the data and use it to set up alerts and draw graphs.
- For a simple and lightweight monitoring solution, DevOps engineers can write a Bash script that automatically makes measurement requests and sends emails if results are not as expected.
Choosing the Right Tool

Want to try network testing platforms but don’t know how to choose the right one for your team? Here’s what to consider:
- Integrations: What integrations does the platform offer? APIs and CLI tools are essential, allowing you to create custom implementations and integrate them with existing tools. Also, look for platforms with user-friendly web-based interfaces that enable non-technical users to run tests.
- Collaboration features: Consider platforms that support collaborative network testing, which is especially important for distributed DevOps teams. For example, Globalping offers a Slack app that brings team members together in a virtual space to solve problems faster.
- Network density: Choose a platform with a dense probe network to ensure reliable measurement results 24/7. You may also want to learn more about the probes themselves. For example, you can review the code to see whether it meets your security requirements if the project is open source.
- Rates & limits: Check the pricing structure and limits carefully, as platforms take different approaches (especially if you need to run custom measurements). Globalping, for example, is a free, open source platform with generous limits that you can extend by becoming a GitHub sponsor. On the other hand, Ripe Atlas uses a “credits” system where you can buy credits or earn them by hosting probes.
- Community & Support: Look for a platform with an active community that supports each other in forums and chat groups. Also, explore the platform’s documentation, tutorials and support channels available to help you get started and assist you in resolving issues along the way.
In summary, choosing a global network testing platform enables DevOps teams to monitor and troubleshoot their infrastructure from anywhere in the world flexibly and cost-effectively. By leveraging the capabilities of these platforms, teams can achieve higher service reliability and a better user experience, even in the most remote corners of the world.
November 10, 2023
Why AIOps is Critical for Networks

Speaker 1: This is Techstrong TV.

Mitch Ashley: With great pleasure of being joined by Andrew Colby. Andrew is VP of AIOps at Vitria. Welcome, Andrew.

Andrew Colby: Good afternoon, Mitch. And thank you.

Mitch Ashley: It’s a great topic. I’m excited to talk with you about it. We could go down the share war stories in telco experience, which really could be about 10 episodes of a different show, but today in the telco environment, or just in the business environment in general, the economic conditions, competitive pressures, looking for areas where we can get more for less, there are a lot of different parameters that have shifted or changed or maybe tightened that we’re currently working within. I’d love to get your perspective on that.

Andrew Colby: Certainly, and thank you. Yeah, I’d say we see cautious optimism. Obviously, I’m based in the US in the DC Metro area, Maryland. And the US, the government entities and quasi-governmental entities have been tightening the economic structure in order to tame inflation. Fortunately, that has not driven our economy and had the potential recessionary effect that was feared, but people are still cautious, businesses are still cautious. That said, it’s hard to hire people and it’s really hard to hire technical people. So a lot of companies are continuing to look towards how to leverage technologies and automation to build efficiency so that they can do more with either the same number of people or re-task their people to higher value purposes, and let the technology do some of the more menial and mundane tasks.

And we can explore this a little bit, especially in these new complex service delivery and network environments. It’s very difficult for me to imagine how an engineer who’s gone through anywhere from two to eight years of college education is going to really be happy going and spending their days collecting a lot of data across network, container management VM and other infrastructure systems to figure out what’s going on. I mean, really that’s where a lot of the automation provides a significant amount of value to let the engineers do the smart, difficult things that we want humans to do.

Mitch Ashley: And a lot of pressures around meantime to recovery, even looking at resiliency, how do we stand up under a test-full situation, whether it be a security attack that might be going on or some unobserved condition that our systems and networks have never been under?

Andrew Colby: Oh, there’s so much of that. So much is changing. It’s not just a person like you or me behind a smartphone that can actually report that there’s a problem, but it’s sensors and equipment that won’t necessarily report right away, so it needs to be detected. So that’s a whole nother additional dimension that service providers, large enterprise IT organizations are under, which is to be able to have this kind of real-time awareness of what’s going on. Whether the service is real time, like the video conference that we’re on or not, there really is a desire and expectation to have real time awareness of the service delivery to be able to detect what’s going on, react to it, address it before the user, whoever that is, the customer, the employee detects a problem, identifies a problem, and actually reports it. That’s really kind of the last line of defense that you want to have.

Now, that said, when your customers are reporting problems, I think it’s really important to have a way to hear that, and that’s another thing that we are able to do with VIA AIOps in particular. We have customers that use that care event data, which is… It’s kind of messy data. It’s human generated, so it’s late in the timeline, but when the users are telling you something’s going on, you need a way to be able to hear that so that you can definitively react. I mean, I know for most people, there’s nothing more frustrating than having an interaction with your technical support teams that tells you, well, there’s really no problem. There’s nothing wrong or I don’t see a problem.

Mitch Ashley: That’s when customers feel like they know more about it than you do, right? I am experiencing this problem, help me please.

Andrew Colby: Absolutely.

Mitch Ashley: Well, I do want to hear some more about VIA AIOps. Just about can’t stumble out the door without tripping over AI in some form, maybe generative AI at the moment. So I’d love to hear some more about your approach around AIOps.

Andrew Colby: Certainly. I mean, it really starts with ingesting a wide variety of information, structured, semi-structured, unstructured information, enriching it, in some cases, classifying it and providing additional context around it. So we take this tremendous amount of data, the telemetry, the faults and others, and by enriching it, we actually grow it even larger and then need to be able to process it in a way that provides meaning for the business. Now, historically, you’d have expert engineers who would say, oh, well this value should be here and above that or below that, would be where we care about it.

There’s two things that are happening simultaneously. There’s too many measures. I was just working with a team turning up a pretty simple system on a container management system, a CMS, and they said, “Well, there’s like 2,400 metrics just in that one system,” haven’t even gotten really to the application measures yet. So tell me what’s important. So that’s one of the challenges, is just you can’t set the values for everyone anymore. And then the other challenge is to know which of them are important. Again, it depends a little bit upon how the service is consumed. Is it one way or bi-directional? Is it real time? Is it latency sensitive? Is it packet loss sensitive? What are the characteristics of it? And depending on those is going to be what’s important and then what’s normal. I mean, every service has some sort of pattern throughout the day and throughout the days of the week.

The machine learning, the unsupervised learning is able to learn what those normals are and then identify when those measures and values deviate from that normal and alert on it. And more important or equally important to just alerting on it, is to be able to bring together all of the related information. And that’s where the enrichment becomes important. It’s not just enough to say, oh, well this measure changed, but well, what is it related to? What are the services that run over that infrastructure? What are the other measures that have changed among those same infrastructure at about the same time? What were the planned changes or manual changes that went in, in the prior 15 minutes? Because maybe one of them had an impact and maybe it didn’t. But that’s really important and useful information to know. So there’s all this bringing together of the enriched information in order to get a full picture of what’s going on.

And that’s a lot of the value of the automation and the AIOps brings together. You don’t need to have your highly skilled, expensive engineers do a lot of that manual data gathering. Let the machine intelligence bring that together for you and let the engineers do the value that humans can do effectively, which is interpret that, identify new patterns, see what’s new, especially if you’re rolling out a new service or you have different behaviors going on. Now, there’s a wide variety of AI and ML that goes into this.

You mentioned generative AI, which is a very interesting topic and has a lot of both technical and popular coverage and attention right now. And there are places where that’s effective to be used for things like the data extraction and ingestion, which is learning what the data is and understanding how to interpret it, as well as potentially developing a suggested fix or likely cause or likely fix that could be generated from all of the different inputs and individual root issues that are identified. But there’s also a variety of other traditional machine learning and AI techniques that really provide a lot of value and are a part of VIA AIOps, and we leverage very extensively.

Mitch Ashley: In a way, I feel like networking, security, they’re very broad topics. In a few words, you’re talking about many, many different technologies. The same is true I think also for AI. There’s so many fields of it. It’s a bit overwhelming to folks. I know you prescribe kind of take an incremental approach. Don’t try to eat the whole elephant at one, if you will, that old adage. But what is that incremental approach? How do you work with customers to start to figure out that path that should go down?

Andrew Colby: That’s a great question. I’d say there’s a couple of steps to that. One is to focus on the business value. What is it that you want to achieve and how can you tell that you’ve achieved it? It’s not just enough to collect a lot of data and produce something that looks a little interesting, aha. Can you do something about it? Does it have an impact on the service? What is the measurable impact on the service? So being focused on business value and even directly measuring it is one important piece. The other is because this is all about network and service operations, it’s generally something that’s done across an entire service or entire company. It’s not so easy to try on maybe a departmental level. So it’s a big decision for companies. Because of the way VIA AIOps is built and structured, we enable it to be delivered or we enable delivery of what we call incremental transformation, which is the ability to augment the existing or augment the machine intelligence in AI with the institutional knowledge.

So to be able to specify policies, for example, for things that are… To differentiate what’s important for the business from what’s important in just a measure, to leverage the value of existing investments. Nobody’s starting this from scratch. So there’s always investments in application performance monitoring or other network and service monitoring tools. They’re not bad, but they just may be sort of siloed or they may have a real key piece of information in one area, and we want to leverage that across the entire service delivery. So that’s another way to provide that incremental transformation and leverage. And overall improving the efficiency of the operation staff and being able to deliver these really as individual use cases incrementally to continue to provide business value over time.

Mitch Ashley: You mentioned automation before, you talked about metrics and things to be what you should be observing. It all starts with getting the data and as you mentioned, bringing, it’s not just correlation anymore, it’s not correlation of events. Yes, we do that or we need to continue to do that, but if you have at 10 factors that are actually part of what’s going on, that’s usually bigger than what someone looking at a monitor or a screen up in an op center is really going to be able to put up together. And this seems like that complexity or maybe the speed of that happening is also a big driver for where you might consider AI. Agree? Disagree?

Andrew Colby: Absolutely. I mean, the kind of highest level metrics are things like the meantime to understand, MTTU, and meantime to restore, MTTR. Those are top level metrics and you want to build down and drive down from those, which is what goes into them. What does it take to understand a problem? In one of our customers, the challenge was not only to understand the problem, but to identify which part of the network the problem was in, so they can get to the right fix agent quickly. Because sometimes you create, in an incident management system, a ticket, but if it goes to the wrong team, they have to triage it, evaluate it, and they say, “Oh, it’s not us. It has to go to…” Maybe it goes to the firewall team instead of the network, instead of the WAN team.

So that’ll take a lot of time. So getting the likely root issue and likely fix to be in the right area is really important to be able to do that. So it’s areas like what are the components that drive that MTTU? How do you measure it? We have a variety of customers and they take a variety of different approaches. In the end, what they want to achieve is a higher percentage of incidents that are handled through automation. And you can do that by decreasing the overall number of incidents as well as by increasing the number that are handled through automation, and we’re able to take both approaches.

Mitch Ashley: It goes back to what you were originally talking about, not being able to hire enough people, the right people, they may not exist, right?

Andrew Colby: Absolutely.

Mitch Ashley: The people we’d like to have is some purple unicorns, but that’s the people don’t always have those skills we’re looking for. So it isn’t always just about downsizing people. Sometimes it’s bringing that curve of what you need to hire right-sizing that just to get… So you can handle the workload with the staff that you have.

Andrew Colby: Right. And some of that is to be able to free up the staff from things like monitoring screens or systems that are just telling you red and green or up or down because that doesn’t have enough context, but you need to understand. And often it’s not just simply red or green, it’s gray or purple, which is, well, it’s working, but it’s not working to the level that we want or need or expect in order to provide the level of service that our customers expect or that our underlying service requires. So it’s being able to provide all that nuance as well as that level of detail and insight, all that enriched information. We go back to that again so that the right action can be taken. Eventually, once the right action gets taken 10 or 50 or 100 times, my expectation is that the trust will have been built in the system so that that action can be now taken in an automated fashion.

And again, that’s an opportunity to accelerate that time to free up an engineer from doing something that they’ve done 98 times before and be able to more quickly allow the action to be taken. Obviously, you’d still maybe want to have some post analysis review to see, well, did we take the right action? Is that same action being taken every day? Maybe there’s some other problem that really we need to address, but still, as long as they can deliver the service in a way that meets the customers’ and expectation and the service expectation, they’re achieving their goal. And they can enable engineers and others’ network operations to kind of free up to think at the higher level about what they need to do and can do to continue to achieve that effectively.

Mitch Ashley: Yeah, we live in a world where it’s very easy for customers to say, your service is taking too long, I’ll jump to the next app or the next site or whatever. There is loyalty, but there’s also patients that tries that.

Andrew Colby: Absolutely.

Mitch Ashley: That trust and loyalty. [inaudible 00:16:54] few minutes left. I’d love to have you talk some more about the incremental approach. I think all the buzz about generative AI, and you hear everything from, we won’t need programmers. We won’t need these kind of people, we won’t need those kind of people. We have robotics in factories, but we still need people in factories, right?

Andrew Colby: Right.

Mitch Ashley: I’m not a subscriber to the… The pendulum doesn’t slam against the other side all of a sudden, those kind of changes don’t happen very often. So it’s probably somewhere in the middle where we end up. So knowing that, if you take that assumption, how do you get aggressive enough that you’re getting some value from AIOps, and I mean so conservative that you’re kind of missing the opportunity?

Andrew Colby: I’d say again, it goes back to aligning your actions with the business values you want to achieve. I’m working with a customer who says they want to achieve a value of running their operations with significantly fewer people, like maybe half or more, less than a traditional network operation center would have in a circuit switch world, let’s say. That’s the goal from the top down. The people that are responsible from the bottom up, they’re like, “Whoa, slow down. Hold on. Don’t do all of your automation yet. We want to look at everything first. We want to see.” Because that’s how they’ve been used to dealing with, and that’s some of the place where there’s tension in being able to do this when the tests are done and you measure response time from issue occurrence to issue detection, to issue resolution, there’s a lot of human think time in there.

And it’s not wasted. It’s people doing their due diligence. Is it really down? Can I restore it myself? Do I need to take an action with an outside team? But those all need to be aligned, and that’s how you can achieve a result. And that’s hard in a large organization. And in all of these cases, we’re talking about large, complex organizations with large, complex service delivery environments.

Mitch Ashley: And you mentioned earlier the trust that’s built in that process too. We don’t just throw AI into the mix and say, it’s in charge now. Let’s walk away and hope it does all right. No, we need to know that-

Andrew Colby: We need to build trust.

Mitch Ashley: … what it’s going to do is the right thing, right?

Andrew Colby: Right. We need to build that trust. And that’s actually one of the reasons that, again, we go back to this incremental transformation. We think that’s really important to be able to do it in steps so that you can see, all right, this is what the system shows and we think the action from that system should be B. So we can give you a button to say, all right, when you believe it’s B, click on B. And again, you do that 50 times. After a while, you’re going to get tired of just saying, well, every time it’s B, why don’t you just do that for me automatically? And that’s where we want to get to.

Mitch Ashley: It’s kind of supervised learning for AI. Very much so. Well, Andrew, it’s been fascinating to talk with you. I hope you’ll get a chance to come back and chat some more. And I really appreciate you sharing sort of on the ground experiences with customers as you’re working in a large setting. It’s one thing to adopt AI in a startup, it’s another in a large telco environment that generally doesn’t make those kind of shifts. On an hourly daily basis, it takes a bit to get that flywheel turning in the new direction that you’re headed.

Andrew Colby: Absolutely.

Mitch Ashley: So, where can folks find out more about VIA AIOps?

Andrew Colby: Thanks, Mitch. Please go to vitria.com on our homepage, and on the Resource tab you’ll find a suite of information about how you can actually realize this business value and the types of capabilities Vitria and AIOps can deliver.

Mitch Ashley: Great. There’s a lot of great resources under that Resource tab, so be sure and check it out. Thanks for joining me again, Andrew Colby, who is a VP of AIOps at Vitria.

Andrew Colby: Thank you, Mitch.

Mitch Ashley: Talk to you again soon.

Andrew Colby: Look forward to it.

October 3, 2023
ngrok Adds Load Balancing to Programmable Network Service

ngrok today added a global server load balancing capability to its service for managing ingress via an application programming interface (API).

Alan Shreve, ngrok CEO, said the ngrok Global Server Load Balancing (GSLB) service reduces latency and increases overall resiliency by automatically rerouting traffic across a global network of points-of-presence ngrok manages.

The overall goal is to provide a higher level of abstraction for managing network operations (NetOps) that can be programmatically invoked. NetOps takes the place of what are essentially the equivalent of Assembly programming tools to provision networking equipment, he added. The long arc of computing is moving toward higher levels of abstraction to make IT capabilities more accessible, noted Shreve.

That approach eliminates the need for IT teams to assign network operations teams to manage appliances in favor of a service that also provides a demilitarized zone (DMZ) via a service that accesses reverse proxies and firewalls, noted Shreve. The ngrok service ensures unauthorized requests are instantly blocked.

The ngrok service provides a programmable approach to managing network services that makes it simpler for DevOps teams to invoke them. Instead of waiting for a network operations team to provision networking equipment, an API makes it possible to integrate the provisioning of networking services within the context of a larger DevOps workflow.

One of the challenges that IT teams continue to encounter is that while virtual machines can be spun up in minutes, the provisioning of network services can require anywhere from minutes to days, depending on the age of the networking equipment. The ngrok service doesn’t eliminate the need for network professionals, but it does make it simpler for them to collaborate with application development and deployment teams, said Shreve.

It’s not clear if NetOps teams will one day simply become an extension of a DevOps team, but as applications become more distributed, the need to minimize network latency is becoming more critical. A service like ngrok makes it simpler for DevOps teams to ensure the appropriate level of bandwidth is available so distributed application performance doesn’t slow to a crawl.

Of course, the bigger challenge will be bridging the cultural divide that currently exists between many NetOps and DevOps teams. Both groups have their own distinct nomenclature for describing services that, in many cases, the other simply doesn’t comprehend. As distributed application development accelerates, each organization will need to determine how to demolish those silos so both teams can more easily achieve a common goal.

In the meantime, IT leaders might to reconsider their strategy as networking services become increasingly programmable via APIs. Most modern networking equipment today is software-defined in the sense that network operations teams have access to graphical tools to provision networking equipment. However, as APIs become more accessible, the need for those graphical tools may not be nearly as high as it is today as networking becomes just another service to be managed as code.

September 14, 2023