devopsy.kinsta.cloud

Tag: containers

Eternity and Infinity

August 12, 2020
Unintended Side Effects

August 3, 2020
The Limits of Technology 2020

July 24, 2020
Snyk Report Finds Decline in Open Source Vulnerabilities

Snyk, a provider of tools for discovering and remediating vulnerabilities in open source code, today published a report that finds the number of new vulnerabilities discovered in open source software packages has declined 20% on a year-over-year basis. However, the bulk of the vulnerabilities being discovered are considered to be high severity, the report finds.

Alyssa Miller, application security advocate for Snyk, said the decline represents a significant improvement as developers become more conscious of cybersecurity issues. As long as humans write code there will always be security issues; however, Miller said the latest report suggests improvements in open source security are likely to continue as more developers embed vulnerability scanning tools within their application development tools.

Those tools enable developers to discover vulnerabilities in their code long before they ever find their way into a production environment, she noted.

The Snyk report is based on a survey of 500 developers, security practitioners and IT operations staff and scans run by Snyk on open source software packages. It finds the top vulnerability currently impacting scanned projects is prototype pollution, which results in objects in JavaScript applications being modified, was found in nearly 27% of all the projects Snyk surveyed.

Better-known vulnerabilities, such as cross-site scripting attacks, are still being reported in high numbers, but Snyk reported the number of projects actually being impacted by this class of vulnerabilities is fairly low. These threats appear to be getting caught and remediated much earlier than many lesser-known vulnerabilities, said Miller.

Want to learn more? Tune in Tuesday, June 30, at 11 a.m. Eastern for The State of Open Source Security in DevSecOps Pipelines: A Panel Discussion, a devopsy.kinsta.cloud webinar hosted by Snyk

The report notes the bulk of the vulnerabilities discovered continue to involve indirect dependencies (70%) within packages built using tools such as Node Package Manager (86%), Ruby (81%) and Java (74%). The survey also finds 60% of organizations do not fully inventory the dependency trees in their software.

Despite the progress being made, Miller said it’s clear much work needs to be done. For example, the survey finds more than 30% do not review Kubernetes manifests for insecure configurations. Requirements for security-related resource controls in Kubernetes are also not widely implemented, according to the survey. Many container images labeled as final contain large numbers of known vulnerabilities—the 14.3 Node image had 642 known vulnerabilities, for example.

In addition, 44% of the participants indicated that they are currently using Kubernetes to orchestrate their containers. Many respondents who said they use Kubernetes also employ Helm to package applications (40%). According to the report, 68% of stable Helm Charts contain an image with a high severity vulnerability.

Ironically, Miller noted many of the same vulnerability issues that have derailed many “golden image” initiatives in the past decade appear to be manifesting themselves again in the age of containers.

Finally, the survey finds 57% of respondents have implemented static code analysis tools (SAST), followed by employing security test cases during quality assurance (28%); dynamic application security testing (DAST) (20%); software composition analysis (19%); and threat modeling (19%).

Despite that relatively low reliance on security tools in the development process, it’s clear progress is being made as DevOps teams and cybersecurity teams collaborate more closely. As more security tools are incorporated into the DevOps pipeline, application security should begin to improve at even faster rates.

June 25, 2020
OpenStack Delivers Ussuri Update

The OpenStack community today officially released Ussuri, an update to the cloud management framework that tightens integration between components including the ability to automatically provision bare-metal servers using the Ironic IT automation framework built for that purpose.

In addition, the community has added IPv6 support to the Kuryr project for integrating OpenStack with container networking and support for the CRI-O image specification for Kubernetes clusters within the Zun container runtime project. As a result, IT teams can now employ Zun-created pods on Kubernetes clusters that can run Kata containers being developed under the auspices of the OpenStack Foundation.

Jonathan Bryce, executive director of the OSF, said support for OpenStack remains strong with more than 1,000 developers from 188 different organizations making contributions to Ussuri. More than 24,000 code changes have been made to OpenStack in this release. According to 451 Research, the OpenStack ecosystem is on track to be valued at $7.7 billion by 2023.

Most OpenStack deployments have been on top of virtual machines. However, as interest in bare-metal servers to run platforms such as Kubernetes has increased, so too has interest in employing OpenStack to automate the provisioning on that infrastructure.

At the same time, the relationship between Kubernetes and OpenStack remains complex. Many organizations prefer to deploy Kubernetes on top of virtual machines to ensure isolation. Others view Kubernetes running on bare-metal servers as a way to eliminate the need for virtual machines altogether. It’s too early to say how that debate will play out but for now most instances of Kubernetes are deployed on virtual machines. The issue is whether to deploy Kubernetes on one of several open source virtual machines or employ commercial virtual machines from VMware that have been widely adopted in enterprise IT environments.

The OSF is also trying to drive the development of Kata containers, which will enable IT teams to employ a container runtime on top of a lightweight virtual machine to achieve isolation between workloads. Bryce said IT teams should expect to see a significant upgrade to the Kata project that will, among other things, make the runtime easier to deploy.

Investments in OpenStack over the last decade have been substantial, so it’s unlikely many organizations that have adopted it are going to abandon it anytime soon. However, in greenfield environments many organizations are considering rolling out next-generation cloud-native applications based solely on Kubernetes. The OpenStack community has already signaled its intent to foster as much interoperability as possible with Kubernetes, which is developed under the auspices of the Cloud Native Computing Foundation (CNCF).

In the grand scheme of OpenStack history, it’s not likely Ussuri will be remembered as a major update. However, it does serve as a reminder of how active the OpenStack community remains, especially as telecommunications providers continue to rely on the cloud framework to automate the provisioning of both 4G and 5G networking services.

In the meantime, enterprise IT teams that have embraced OpenStack as well should expect to be employing it to provision infrastructure to run both monolithic and microservices-based applications for years to come.

May 13, 2020
Decoding the Self-Healing Kubernetes
A business application that fails to operate 24/7 would be considered inefficient in the market. The idea is that applications run uninterrupted, irrespective of a technical glitch, feature update or a natural disaster. In today’s heterogeneous environment where infrastructure is intricately layered, a continuous workflow of application is possible via self-healing.

Kubernetes, which is a container orchestration tool, facilitates the smooth working of the application by abstracting machines physically. Moreover, the pods and containers in Kubernetes can self-heal.

In the Avengers movie, Captain America asked Bruce Banner to get angry so he could transform into The Hulk. Bruce replied, “That’s my secret Captain. I’m always angry.” You must have understood the analogy here. Let’s simplify: Kubernetes will self-heal organically, whenever the system is affected.

Kubernetes’s self-healing property ensures that the clusters always function at the optimal state. Kubernetes can effectively self-detect two types of objects—podstatus and container status. Kubernetes’s orchestration capabilities can monitor and replace unhealthy containers as per the desired configuration. Likewise, Kubernetes can fix pods, which are the smallest units encompassing single or multiple containers.

The Three Container States Include
1. Waiting: created but not running. A container, which is in a waiting stage, will still run operations such as pulling images or applying secrets, etc. To check the waiting pod status, use the following command:
  
  Along with this state, a message and reason about the state are displayed to provide more information.
2. Running Pods: containers that are running without issues. The following command is executed before the pod enters the running state:
  
  Running pods will display the time of the entrance of the container.
3. Terminated Pods: a container that fails or completes its execution. The following command is executed before the pod is moved to terminated:
  
  Terminated pods will display the time of the entrance of the container.
Kubernetes’ Self-Healing Concepts – Pod’s Phase, Probes and Restart Policy

The pod phase in Kubernetes offers insight into the pod’s placement. We can have:
- Pending Pods–created but not running.
- Running Pods–runs all the containers.
- Succeeded Pods–successfully completed container lifecycle.
- Failed Pods–minimum one container failed and all containers terminated.
- Unknown Pods.
Kubernetes execute liveness and readiness probes for the pods to check if they function as per the desired state. The liveness probe will check a container for its running status. If a container fails the probe, Kubernetes will terminate it and create a new container in accordance with the restart policy. The readiness probe will check a container for its service request serving capabilities. If a container fails the probe, then Kubernetes will remove the IP address of the related pod.

Liveness probe example:

The probes include:
- ExecAction–to execute commands in containers.
- TCPSocketAction–to implement a TCP check w.r.t to the IP address of a container.
- HTTPGetAction–to implement a HTTP Get check w.r.t to the IP address of a container.
Each probe gives one of three results:
- Success: The container passed the diagnostic.
- Failure: The container failed the diagnostic.
- Unknown: The diagnostic failed, so no action should be taken.
Demo Description of Self-Healing Kubernetes

We need to set the code replication to trigger the self-healing capability of Kubernetes.

Let’s see an example of the Nginx file:

In the above code, we see that the total number of pods across the cluster must be four.

Next, let’s deploy the file.

Let’s list the pods, using:

Here is the output:

As you see above, we have created four pods.

Now, let’s delete one of the pods.

The pod is now deleted. We get the following output:

Let’s list the pods again.

We get the following output:

We have four pods again, despite deleting one. Kubernetes has self-healed to create a new node and maintain the count to four.

Conclusion

Kubernetes can self-heal applications and containers, but what about healing itself when the nodes are down? For Kubernetes to continue self-healing, it needs a dedicated set of infrastructure, with access to self-healing nodes all the time. The infrastructure must be driven by automation and powered by predictive analytics to preempt and fix issues beforehand. The bottom line is that at any given point in time, the infrastructure nodes should maintain the required count for uninterrupted services.
May 12, 2020
5 Capabilities Network Infrastructure Must Have to Support DevOps

According to the Accelerate 2019 State of DevOps report, high-performing DevOps organizations achieve over 200 times more frequent deployments and over 2,600 times faster recovery times than low performers. These outcomes have been realized by both forward-leaning organizations such as Amazon, which deploys code every 11.6 seconds, and more traditional enterprises that have been in existence for a long time. For instance, Nordstrom, a well-known American retailer, increased the release frequency for its mobile app from twice per year to monthly.

Embracing DevOps results in significant advantages, including improved team collaboration, cooperation and productivity. Companies can get digital products, applications and services to market faster because of the higher deployment and release frequency. Because development and operations teams are not operating in silos, unaware of each other’s constraints, there are lower failure rates within production environments. Cooperation leads to faster recovery times when there are issues with deployment.

To achieve DevOps success, teams must engage in a new approach to network infrastructure, and the network infrastructure must have the following five attributes.

Automation

Automating repetitive tasks, such as configuring DNS or a firewall, saves a lot of time and effort. It eliminates manual errors by standardizing deployment and configuration of network infrastructure. Automation can be achieved by using Infrastructure as Code (IaC), which entails using an application programming interface (API) to create, modify and shut down infrastructure. This ensures that a repeatable, consistent configuration can be created. This code can be version-controlled, which serves as a form of documentation—it’s easy to detect what changes were made by whom in case there’s an issue—and rollback to a stable state is quick and easy.

Standardization puts an end to “It worked in my environment” or “It worked on my machine” as any issues can be easily reproduced, resulting in faster resolution times. Support for IaC tools, such as Terraform, is essential for network infrastructure to advance DevOps practices.

Continuous Integration/Continuous Delivery (CI/CD)

CI is a coding best practice wherein code is checked into version control repositories frequently. It provides a consistent and automated way to build, package and test applications. CD picks up where CI ends and deploys applications in an automated fashion in a variety of environments, including production, development, test and sandbox.

There are a variety of tools, such as Jenkins and Ansible, that DevOps teams use for CI/CD integration. It is imperative for network infrastructure to integrate with these tools to accelerate deployment velocity.

Elasticity

DevOps teams spin up and spin down test/development/sandbox environments routinely. Infrastructure may also be needed for short periods to account for seasonality or for handling sudden surges in traffic and demand. DevOps-friendly network infrastructure must be elastic to support these activities.

Appliance-based vendors who provide network services are not scalable as they impose artificial limits on their usage (such as queries per second or leases per second) and require organizations to anticipate their needs in advance. When there is a need, organizations have to spend time procuring these appliances, waiting to receive them and, finally, deploying and managing them—a very inefficient way to manage demand.

Support for Containers

Containers are the foundation for developing cloud-native applications. Containers allow developers to package up application code and all its dependencies, including libraries and other binaries, as well as configuration files needed for running the application. This enables applications to be executed quickly and reliably across a variety of environments.

Containers provide flexibility and portability of applications across public, private, hybrid cloud and even bare-metal environments. They are lightweight, as all containers share the underlying machine’s OS system kernel. They also lend themselves to greater modularity, which is essential for developing microservices-based applications.

Instead of a monolithic application consisting of a three-tiered architecture, a microservices-based application consists of several smaller services that perform one function and one function only. It’s easier to scale, easier to manage and faster to develop such applications. Because containers have a small footprint, they are most suited for microservices architecture.

In order to support containerized applications, network infrastructure must first be compatible with containers. Network infrastructure must be able to configure DNS, allocate IP addresses for containerized applications, route traffic intelligently between containers and balance loads for optimal performance. Network infrastructure must also support Kubernetes, the most widely deployed orchestration platform. To ensure that the needs of DevOps teams are met, it is important to inquire with network services vendors about compatibility with containers and Kubernetes, as many legacy, appliance-based solutions cannot support this technology.

Self-Service Access

To move quickly, development teams may need to configure network infrastructure for test/development/sandbox environments. Having to submit tickets to network operations teams that use legacy appliances for delivering network services significantly slows down application development and deployment. Providing self-service access to network infrastructure, such as firewalls, load balancers, DNS and DHCP for development teams, eliminates the reliance of these teams on network operations teams. Self-service with robust role-based access controls reduces the deployment time and increases efficiency while freeing up network teams for more strategic projects.

Conclusion

DevOps attempts to minimize friction in application deployment by providing autonomy to development teams so that they don’t have to rely on network administrators to provide the necessary infrastructure needed for development and testing. But, the move to DevOps requires more than a consolidation of teams.

Success requires best practices such as automation, CI/CD and self-service access to infrastructure, as well as support for containers and Kubernetes, which are key to building modern, cloud-native applications. Teams that incorporate these capabilities will see increased agility and speed that lead to shorter delivery time, higher-quality applications and improved communication and collaboration throughout a product’s lifecycle.

May 1, 2020
Archived

April 17, 2020
In IT, Purity Is Not Necessarily a Virtue

Throughout the history of IT, we’ve had our share of purists. OS purists, language purists, etc. The stark one was the “That is not Object Oriented Programming” crowd. While the poster-children of IT purity, we don’t really hear from them anymore—and thank goodness for that.

But a new crowd has reared its head in DevOps specifically, development more generally. They don’t believe there is ever a reason to modify a running container. Call them deployment purists, though maybe one of you has a better term.

Like OO purists, this approach ignores the reality of the environment we work in. Yes, the best option most of the time is to fix an issue and roll out a replacement container. But, the fact is that claiming anything else is not using containers “correctly” is patently wrong. My go-to example for why I say this is the easy one, because explaining complex infrastructures while trying to make your point is a waste of time.

Rolling feature releases in most organizations that do them regularly are achieved with environment variables on feature tags. But changing an environment variable is changing an instance. So, are all these organizations wrong, or perhaps the idea of purity is the problem when it comes to IT? Certainly, the driver for DevOps in many organizations was that purity had created islands of toolsets with varying delivery timelines and quality controls.

I contend it is past time to let purity be a guideline instead of a mantra. Yes, in most situations rolling out a new instance is the correct response, but in many–live applications that will drop connections and lack enough redundancy to bleed users off, for example–the correct solution is to modify the instance. “Never change production” is a good mantra, but it is a terrible purity test. We should avoid it, while acknowledging that not all situations allow us to make that choice.

Keep on rocking it. We’re weeks into COVID, and I can hear the stress in my operations friends’ voices. Know that the rest of us are grateful to have you keeping things running for us and bending over backward to make certain we have the access we need. Thank you, from me, for all of the people relying upon you.

— Don Macvittie

April 14, 2020
Threading the Needle on Kubernetes Complexity with AI-Powered Observability

If you run down the laundry list of major industries–transportation, retail, healthcare, automotive, financial services–you’ll see that the major players in each have essentially become software companies. What they do and how they serve their customers rests entirely on their software working perfectly. Underpinning this is a massive digital transformation that has shifted organizations away from the old, static ways of enterprise IT monoliths. Instead, today’s businesses are working in more dynamic IT environments–agile, multi-cloud and, especially for new digital services, based on containers and microservices. All of which means embracing more IT complexity than ever before.

Kubernetes Brings More Data to the Table, but that Impacts Observability

A key part of this more complex environment is the soaring adoption rate of Kubernetes over the last few years. In a survey of 5,000 enterprises, 40% said they were currently running Kubernetes in production and over 80% reported that they’d adopted Kubernetes over any other container management system. Another recent survey revealed that 68% of CIOs are already using containers and a total 86% expect to deploy them over the next year. Enterprise IT is increasingly gravitating to containers and especially to Kubernetes for orchestrating their containerized applications.

What drives organizations toward containers and Kubernetes is the flexibility given to development teams to release services, features or apps faster. But faster doesn’t necessarily mean better or higher quality, unless you also apply cloud-native architectural patterns, include quality gates into your delivery pipeline, embrace new deployment models such as canary or feature toggling and provide full visibility and short feedback loops from production back to engineering.

Kubernetes itself provides great visibility into node or pod health. Kubernetes even ensures the health of your pods through its built-in health check mechanisms. Thanks to components such as service meshes, you also get live information about dependencies between services running in your pods. This data can, in theory, provide organizations with the necessary insights into their environments to understand whether their organization is releasing better value faster to their end users and unlocking new business value.

The reason I said these are theoretical benefits is because, in practice, all that extra data doesn’t necessarily mean more visibility. Often it can mean less visibility–and with that, less insight into how the different elements of your environment may be impacting speed of production deployments, overall system performance and the end-user experiences. I’ve seen Kubernetes environments that are packed with literally billions of interdependencies between applications, containers, microservices and clouds. Asking human beings to keep track of that level of data manually is an impossible task now. And relying on traditional, manual approaches to manage and monitor IT environments shortchanges the value that organizations should be getting out of Kubernetes.

Kubernetes Environments Need Full Stack Observability—that Means Needing Deterministic AI

Getting the full potential out of Kubernetes means having observability across the full technology stack and those billions of interdependencies. But as more organizations are discovering, that can’t happen without AI. Specifically, a deterministic AI engine. In fact, in a 2019 survey, 88% of CIOs said they were looking to AI as a critical solution for overcoming IT complexity. As long as Kubernetes is fueling complexity, overcoming that complexity means making deterministic AI an integral part of any Kubernetes environment.

A deterministic AI applies specific algorithms to detect system anomalies and apply its collected dependency information toward detecting the root causes behind those anomalies. When that deterministic AI engine ingests critical Kubernetes events, such as state changes and workload changes, it provides visibility into the Kubernetes stack’s interdependencies. That AI-powered, full stack observability matters because it gives IT new capabilities for their environments, including instrumenting and mapping containers, managing and allocating container resources and drilling down into container runtimes at work. All of that is possible due to the level of observability that you can only get with deterministic AI; traditional metrics and dashboards just can’t compete.

By providing full stack visibility and interdependency mapping across a Kubernetes environment, deterministic AI is empowering IT teams to identify, in real time, the root cause of degradations in business service availability–from reliability and resource usage to system performance and user experience–faster than they ever could manually. That level of automated speed and visibility also makes it possible to immediately resolve those issues at the source before they can impact the end user. The automation provided by deterministic AI also works its way into DevOps’ production pipelines, accelerating the release and feedback cycles of new deployments and allowing teams to more quickly incorporate user feedback into the next iteration for even better experiences.

In short, a deterministic AI engine makes life better for both IT and the customers they’re aiming to serve. Deterministic AI takes work that used to be time-consuming, tedious and counterintuitive when done manually, and performs it fast and automatically. That then frees up the teams that used to do that to focus on work that is more satisfying and drives new business value.

AI-Powered Observability Relies on Quality Data and Quality Monitoring

All that said, a deterministic AI’s ability to provide reliable, precise full stack observability in a Kubernetes environment relies on it being given consistently quality data it can process. The algorithms are dependent on the data that’s ingested by the AI engine. If there’s no standard monitoring capability being used across the board, then you invite a scenario where different teams bring their own preferred monitoring tools into the fold. The bigger a Kubernetes cluster gets, the more diverse the set of monitoring tools can be. And the greater the diversity of monitoring tools, the more inconsistent the data being fed into the deterministic AI engine.

The way out of this predicament is to leverage a single monitoring tool as a self-service capability into Kubernetes clusters, one that can support the diversity of technologies that are introduced into the stack and provide a baseline standard of system and event data that developers, IT administrators and the deterministic AI all lean on. In other words, when you feed consistent, quality data into the AI, you make it smarter. And a smarter deterministic AI engine provides the full stack Kubernetes observability that organizations need.

Building a Positive Feedback Loop Between Kubernetes, Deterministic AI and Observability

Kubernetes and deterministic AI are a natural fit. Kubernetes provides a platform for managing and instrumenting containerized applications. The deterministic AI provides a tool for mapping out the Kubernetes stack, offering insights into interdependencies and degradations where they arise and resolving them before they can affect end users. The data from Kubernetes makes AI better, and a smarter deterministic AI engine means more precise observability across your technology stack.

It’s a self-reinforcing feedback loop that eliminates the usual pitfalls of container-induced IT complexity. Instead, it creates new standards of visible, consistent and quality data that organizations can leverage for faster and better deployments–and in turn, better user experiences and greater business value.

To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon EU, in Amsterdam. The CNCF has made the decision to postpone the event (originally set for March 30 to April 2, 2020) to instead be held in July or August 2020.

— Andreas Grabner

March 23, 2020