devopsy.kinsta.cloud

Author: Manuel Zapf

  • Running Kubernetes in Production: Make Sure Your Routing Strategy Works

    Running Kubernetes in Production: Make Sure Your Routing Strategy Works

    Managing Kubernetes deployments in production raises some rather complex challenges. The dynamic nature of network communication in containerized environments produces unique operational issues–capable of bewildering even experienced DevOps teams. To mitigate these issues, software-based routing components offer key advantages across a number of areas.

    Here are a few particularly important routing strategies to consider within any production-grade Kubernetes environment.

    Leverage Tracing and Monitoring

    However thoroughly you test applications during development, a fresh round of issues will emerge in production. To help understand (and respond to) these new obstacles, tracing and monitoring tools provide developers with crucial visibility into their runtime Kubernetes environment. Choosing routing technologies that closely integrate with established monitoring and tracing backends will also help make things easier.

    All inter-service traffic passes through software routing components. This enables strategic application designs that purposefully collect microservices tracing and monitoring data. For example, tracing tools can identify the source and call flows of microservice invocations (assuming that the application is designed to support traceability). Thus, teams can determine how to fully leverage any available microservice, even those developed by different teams. By designing applications to provide metrics, DevOps teams can also use software routing and monitoring tools to understand production issues. For instance, tracking resource usage metrics for a microservice can reveal if its subcomponents are prepared to handle load at greater scale. That data can simultaneously point out performance limits and aid in troubleshooting new feature implementations.

    Secure Communications and Permissions

    Security must be a top-of-mind focus in any production Kubernetes deployment. Permissions governing internal network communications need to be tightly controlled to defend against attacks. Sensitive data transmitted within internal and external traffic must be encrypted and secured. Many businesses require data encryption not just as a best practice, but also as a matter of regulatory compliance. Unfortunately, these safeguards are absent in many Kubernetes environments.

    Routing technologies make it possible to enforce security policies using network segmentation. For example, only client services with justified business requirements should have access to microservices that handle sensitive data. Routing tools can also provide encryption. Service meshes can secure internal east-west traffic with secure TLS encryption. Edge routers can utilize provided certificates to encrypt all external north-south traffic. In order to fully automate the lifecycle management of trusted certificates, DevOps teams can also pair up routing technologies with services such as Let’s Encrypt. This automation makes it possible to continuously encrypt and secure the transport of sensitive data, with no interventions from human workers required.

    Absorb Unexpected Load Spikes

    Sudden popularity, DDoS attacks and other unexpected events can trigger surprise load spikes. As a technique, rate limiting enables operators to control the request rates to front-end services. This equips applications to absorb surprise load spikes and avoid failures. In this way, leveraging a routing tool to implement rate limiting techniques effectively limits downtime due to load events.

    Resolve Communication Issues

    Microservices can become unreachable due to errors by a container, host, network partition or short-term microservice interruption. Therefore, implementing mitigation strategies to keep these communication failures from affecting users is absolutely crucial.

    For example, the load balancer can respond to instance failures by directing requests to viable instances (and resuming normal traffic when the instance is available). During microservice interruptions, it’s critical to halt repeated retry requests to conserve resources and avoid cascading failures. Teams should use client services to employ circuit breaking to stop requests, send errors and fallback to alternative procedures.

    While it’s possible to implement these mitigation techniques using network-layer logic within each client, that approach is highly challenging and error-prone. Teams are better off utilizing routing technologies to control their applications’ mitigation responses.

    Achieve High Availability

    Compared to traditional hardware failover solutions, today’s software routing technologies offer a far more effective and cost-efficient means of introducing high availability to Kubernetes environments.

    Software routing technology enables DevOps teams to leverage architecture via a separate, horizontally-scalable data plane and a fault-tolerant control plane. The data plane makes it possible to add all the instances necessary to achieve the requisite capacity and resilience. At the same time, the control plane effectively tolerates failures to preserve uptime and safeguard seamless experiences for an application’s user base. A smart routing strategy thus ensures that production Kubernetes deployments are highly available and incredibly resilient.

    Deploy Across Multi-Cloud and Other Heterogeneous Environments

    DevOps teams may leverage Kubernetes across multiple cloud environments, on-prem environments and even alongside other container orchestration solutions. A routing strategy should meet the portability required to effectively support deployments across these different environments and solutions. By doing so, developers can deploy a common routing layer model across all deployments. This reduces key challenges by enabling a concentrated focus on a single and familiar solution.

    Selecting an appropriately advantageous software routing technology can transform the experience of deploying Kubernetes into production. Routing tools provide capabilities that can simplify or even fully eliminate many common challenges (and some uncommon ones). In this way, DevOps teams can take advantage of everything Kubernetes has to offer that much more easily and successfully, while achieving more robust and reliable applications.

    To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon EU, in Amsterdam. The CNCF has made the decision to postpone the event (originally set for March 30 to April 2, 2020) to instead be held in July or August 2020.

    — Manuel Zapf

  • Six Routing Challenges When Managing Microservices in Production and How to Avoid Them

    Six Routing Challenges When Managing Microservices in Production and How to Avoid Them

    Even experienced DevOps teams can struggle with the nuances of managing microservices in production–where a greater spotlight on network communications can introduce new operational challenges beyond those common to traditional architectures. But, by being smart about your software-based routing components, it’s possible to avoid or reduce the impact of roadblocks in the following areas.

    Implementing Tracing and Monitoring

    Even applications that are rigorously tested during development will exhibit anomalies in production. To address these, DevOps teams require effective tracing and monitoring tools to gain visibility into what’s happening within clusters at runtime.

    As a strategy, developers should select routing solutions that integrate well with their existing monitoring and tracing backends. Remember that while a microservice may be developed by a certain internal team, once deployed it becomes available for others. If the application is instrumented to be traceable, compatible tracing tools can dynamically identify the source of service invocations and their call flows–enabling other teams to utilize the microservice. When the application is also designed to provide metrics, monitoring tools can usefully track the microservice’s key resource consumption metrics to assess the scalability of its subcomponents under load, and recognize any performance limitations that require attention.

    Because all inter-service communications travel through software routing elements, you can strategically design your application to enable microservices tracing and monitoring data to be collected. Teams can then put this data to work to more easily overcome challenges when building out new capabilities.

    Mitigating Communication Failures

    Considering the myriad communications taking place within production microservices-based environments, it only makes sense that this architecture will be prone to communication failures in which microservices simply cannot be reached. Whether these failures occur as the result of an error by the container, host, a network partition, or a short-term interruption in the availability of the service itself, resilient safeguards must be in place to mitigate any impact to an application’s user base. 

    For example: When an instance fails, the load balancer can be used to automatically reroute requests to healthy instances, and then return traffic to the failed instance upon detecting that it’s again available. In scenarios where an entire microservice becomes unavailable, client services should leverage circuit breaking to stop requests, return an error and initiate a fallback response. Doing so avoids dangerous repetition of retry requests, which only eat up resources and can lead to cascading failures.

    Each of these mitigation strategies could be accomplished by placing the necessary logic with each client. However, this technique would be both cumbersome to implement and prone to errors. The better approach to these challenges is to leverage routing solutions to perform instance health checks and circuit breaking, allowing DevOps teams to control how their applications respond to faults, rather than relying on logic at the network layer.

    Handling Unanticipated Load Spikes

    In production environments, loads can spike unexpectedly and for any number of reasons–ranging from simple sudden popularity with users to malicious distributed denial-of-service (DDoS) attacks. Whatever the root cause, applications must be well-prepared to withstand these events without failing.

    To ready applications to handle load spikes, rate limiting is a simple (but powerful) technique that allows operators to specify the request rates to front-end services. Implementing this safeguard through a routing solution answers the challenge of avoiding downtime due to surprise changes in load intensity.

    Securing Microservices Communications and Permissions

    Security is always a constant concern when it comes to operating production deployments. However,  microservices environments often have absolutely zero enforcement of internal network communications permissions. They also face the challenge of implementing proper encryption to secure sensitive data within both internal and external traffic. 

    While allowing any service to access any other offers flexibility, the risks are substantial. In order to safeguard microservices that access and expose sensitive data, it’s wise to limit access to only those client services with a vetted business justification. Routing technologies make it possible to enforce these secure policies using network segmentation.

    At the same time, routing solutions can help businesses encrypt the in-flight transport of critical data, which is not only a best practice, but in many industries a necessity from a regulatory compliance perspective. Service meshes can offer the benefit of providing secure TLS encryption for all internal east-west traffic, while edge routers offer to encrypt external north-south using a provided certificate. Also of note for north-south traffic: combining routing technology with services such as Let’s Encrypt can serve to fully automate lifecycle management of trusted certificates, eliminating the need for human intervention entirely.

    Achieving High Availability

    Leveraging software routing technology to achieve high availability is more cost-effective and efficient than traditional hardware failover solutions. It offers the opportunity to build production microservices environments that are quite resilient. For example: Routing implementation should utilize a highly-available architecture that includes a horizontally scalable data plane and a separate fault tolerant control plane. While the data plane allows for adding instances as necessary to meet capacity and resilience needs, the control plane is prepared to tolerate failures and ensure that users experience seamless and uninterrupted uptime.

    Deploying Across Multi-Cloud and Heterogeneous Environments

    Larger enterprises often utilize deployments across multiple cloud and/or on-prem environments, each of which may rely on disparate container orchestration technologies. Highly portable routing technologies capable of supporting a range of orchestration tools can vastly reduce the challenges developers face in supporting multiple environments. This allows them to deploy the same familiar solution and a common routing layer model across deployments.

    Routing technology is a key variable in determining the capabilities and ease with which DevOps teams are able to overcome challenges to the success of their production microservices deployments. By making a carefully considered choice in selecting a routing solution, teams can better prepare for these challenges–or even avoid them altogether.

    — Manuel Zapf