devopsy.kinsta.cloud

Author: Jason Bloomberg

  • Cloud-Native Security and Performance: Two Sides of the Same Coin

    Cloud-Native Security and Performance: Two Sides of the Same Coin

    You’re running Kubernetes in a production environment, and you need to apply a patch — perhaps to a commercial application, an open source component or even a container image. How long should it take to implement that patch in production? Thirty days? One day? One hour?

    Remember, cloud-native environments are supposed to respond to change in real-time. Such a response isn’t simply scaling up or down as needed. It’s also essential to respond to security threats, and performance issues, as close to real-time as possible.

    Containers, and the microservices they support, are also ephemeral. They can live for five minutes or even less, and yet unpatched microservices are just as dangerous as more permanent code.

    Patching must also fit within your application lifecycle. Teams with a mature DevOps process typically release code weekly, daily or even hourly. Their ability to apply the patch and roll to production quickly, significantly reduces their security risk. Are your operators up to the task?

    Poor Security Management Becomes a Performance Problem

    The connections between security and performance go well beyond cloud-native environments. Denial-of-service (DoS) attacks obviously target a site’s performance, as do cryptojacking and to a lesser extent, ransomware attacks.

    In many cases, performance monitoring gives an organization its first indication that such an attack is underway. However, monitoring isn’t just for recognizing attacks in progress. It also plays an important role in the prevention of attacks as well — especially in cloud-native environments.

    To understand this point, think about how organizations have traditionally handled patch management, especially to software infrastructure: A vendor (or open source project) releases a patch. The enterprise applies the patch in a test environment that resembles production somewhat. After running a range of integration tests over the course of days or even weeks, the ops team may be ready to deploy the patch into production. 

    Another likely scenario: The ops team may collect several patches to different pieces of software, hoping to test and deploy them all at once. In the meantime, more time goes by, giving attackers even more opportunity for mischief.

    The reason that applying patches in an enterprise production environment takes so long is because IT leadership perceives that patch management is a high-risk activity. Patches may make a piece of software behave poorly or not at all — and the complex interdependencies among both applications and infrastructure components compound the risk of failure unpredictably.

    Cloud-Native Computing Changes the Equation

    Cloud-native computing extends cloud best practices such as scalability and resilience to the entire enterprise IT landscape, providing a new paradigm for computing across hybrid IT environments that leverage virtualization, containers and serverless computing as appropriate.

    At the center of the cloud-native movement is the open source container orchestration platform, Kubernetes. Kubernetes requires and reinforces a cloud-native architectural approach that decomposes applications into containerized microservices. 

    Deploying, managing and operating microservice-based workloads at scale becomes the central challenge for Kubernetes, and for cloud-native infrastructure in general.

    To manage patches, organizations must follow cloud-native principles. Monolithic, “test everything for weeks” approaches simply do not align with the dynamic, ephemeral nature of cloud-native software.

    Just as application developers use canary testing for features they are rolling out, canary testing infrastructure updates largely supplants testing in the full-blown test environment. 

    Testing patches (or updates) become an ongoing, largely automated process of trying new configurations in limited production environments, analyzing the resulting performance and rolling back changes if necessary.

    Operators can now test patches (or new releases) more quickly in production, in a way that ideally minimizes the impact on the applications that end-users interact with.

    How, then, should operators manage such canary environments? By monitoring the performance of the software under test.

    The Convergence of Security and Performance Monitoring

    Unpatched software is the leading source of vulnerabilities in the enterprise IT landscape, and managing patches in dynamic environments requires performance monitoring.

    Add patch management to other performance-related cyberthreats, such as denial of service, ransomware and cryptomining, and it’s clear that security and performance monitoring are two parts of a broad spectrum of IT operations management (ITOM) capabilities.

    In cloud-native infrastructure, ITOM doesn’t stop at monitoring. Instead, operators require observability tools that monitor logs, traces and metrics across the entire infrastructure, in addition to providing tools that operators can use to mitigate and prevent problems with the infrastructure.

    In other words, where monitoring is a passive, “watch the dashboard” activity, observability goes the extra mile, promoting an active, “understand and fix the problem” mentality.

    It doesn’t really matter if those problems are performance or security-related — in many cases, there’s no distinction. Today’s operators must work with tools that bring the two together into a full-fledged security, compliance and monitoring solution.

    The Intellyx Take

    Some analysts argue that IT infrastructure has followed a natural progression from on-premises servers to virtualization to cloud computing to containers. However, there is a fatal flaw in this argument.

    The first three environments in this sequence follow a clear pattern, as virtual machines work much the same as physical servers, be they on-premises or in the cloud. As a result, coding applications remained largely unchanged as enterprises worked their way along this progression to the cloud.

    Containers, however, are fundamentally different — especially once you place them into Kubernetes and the cloud-native context overall. With cloud-native computing, it’s essential to rethink the nature of the application from the ground up. Cloud-native computing is, in fact, an entirely new paradigm for enterprise IT.

    It’s no wonder, therefore, that core operational activities like security and performance monitoring have a new role in the cloud-native world. Indeed, security and performance are every bit as important as before, but today, we must deal with both at once in the context of a dynamic, ephemeral application landscape.

    Only within this context does it become clear what cloud-native vendors like Sysdig are offering. The world of IT infrastructure is changing, and with it the world of enterprise application development and deployment. Security and performance are as important as ever. Don’t let them fall through the cracks.

  • DevOps Versus ITIL: How to Win the Battle Over Change Management

    DevOps Versus ITIL: How to Win the Battle Over Change Management

    Ever since DevOps came on the scene a decade ago, ITIL has been under siege.

    ITIL lays out a number of IT service management (ITSM) best practices, including release management, change management, incident management, problem management and many other aspects of running an IT shop so that its priorities align with the business’s. For many enterprises, ITIL is essential for governance and regulatory compliance–especially in heavily regulated industries.

    The basic idea of DevOps is that by leveraging better automation technology across the software lifecycle, it’s possible to rework the culture and organization of all the teams involved in software production and deployment. By breaking down the numerous silos and fostering greater collaboration, it should be possible to dramatically accelerate the ability to deploy quality software at scale.

    From the DevOps perspective, ITIL is nothing but a set of speed bumps and roadblocks, getting in the way of its continuous integration/continuous delivery (CI/CD) priorities.

    DevOps wants to speed up software release cycles, while change management often slows them down. Does this dichotomy mean that enterprises have to choose one or the other? Or perhaps we have to water down one of these approaches sufficiently so it’s marginally compatible with the other one?

    Is ITIL Part of the Problem?

    ITIL skeptics abound. “Frameworks like ITIL create the illusion of control that results in [the] organizational delusion that IT is efficient and manageable,” said Greg Ferro, long-time consultant and co-founder at Packet Pushers Interactive. “The single biggest problem in ITIL [is that] the creation of silos creates ‘not my problem’ attitudes.”

    One such silo is the change advisory board (CAB), a team that has the responsibility for making thumbs-up or thumbs-down decisions on numerous different activities across the IT organization. “Having the CAB review every single change request isn’t efficient,” said Kaimar Karu, head of ITSM at AXELOS, “and it’s definitely not common sense, especially when their costs can run to tens of thousands of deployments per hour in some organizations.”

    One organization that has struggled to transform their ITIL process as it moved to a DevOps model is financial services firm ING. In fact, its CABs were a point of contention for it. “We were just like an insurance company, always rejecting your first claim,” said Mark Heistek, IT specialist at ING. “And then come back if you have good arguments.”

    ING found it was able to modify ITIL to work within its new DevOps approach. “Don’t do everything the ITIL book says,” advised Jan-Joost Bouwman, ITSM process owner at ING. And yet, following ITIL for practices such as incident management are “still the best way to do it, because everybody knows what to do and you don’t get confused about the rules.”

    Or Is ITIL Part of the Solution?

    In spite of the skepticism, there are plenty of proponents of using ITIL with DevOps. “The DevOps Movement fits perfectly with ITSM,” according to Gene Kim, DevOps thought leader, author of The Unicorn Project and coauthor of The Phoenix Project. “ITIL and ITSM still are best codifications of the business processes that underpin IT operations, and actually describe many of the capabilities needed into order for IT operations to support a DevOps-style work stream.”

    Kim, however, focuses more on human capabilities than on the ITIL processes and procedures themselves. “ITSM practitioners are uniquely equipped to help in DevOps initiatives, and create value for the business,” said Kim. “ITSM skill sets are ever more important in a world where there is an ever-quickening business tempo.”

    Phil Tee, CEO and co-founder of Moogsoft, agrees with Kim–and also echoes the experience of Heistek and Bouwman at ING. “ITIL is not in conflict with DevOps,” said Tee, “but it must be adapted to DevOps practices in order to define clearly where issues arise and how to solve problems efficiently.”

    Adapting ITIL to work within the context of DevOps–without slowing it down–has become the crux of the discussion. “ITIL is not explicitly opposed to Agile and DevOps,” explained Charles Betz, principal analyst at Forrester Research. And yet, he found that “the overall ITIL narrative is still sequential, plan-centric and deterministic.”

    Rock and a Hard Place

    Part of the DevOps way of thinking is to reevaluate how the organization makes the decision to deploy software into production. “By combining developers and operations into smaller, more agile DevOps teams, organizations can build software more quickly than ever, enabling them to respond faster to changes in the market,” said RJ Jainendra, general manager and vice president, IT business management and DevOps business unit at ServiceNow. “But, that’s a difficult challenge for larger companies, however, that haven’t been able to realize the full benefits of DevOps for a variety of reasons, including the complexity of manual processes they manage, and a shortage of DevOps expertise.” 

    There are many business contexts where internal process or regulatory compliance concerns are the issue, introducing requirements beyond what current DevOps approaches can satisfy.

    Jumping to the opposite extreme and requiring rigid change management procedures, however, can easily be overkill. After all, even for the most heavily regulated, quality-conscious enterprise, some of the changes they put into production don’t have the same stringent requirements as the rest. Not all software changes carry the same level of risk.

    The challenge, therefore, is telling the difference–and putting into place proactive, automated ways of handling different change scenarios that are subject to different levels of rigor.

    Judicious Use of Automation

    In spite of what many DevOps zealots may say, the approach is inherently flexible and human-driven. People still create software, after all–and DevOps is more about encouraging new ways for people to cooperate to achieve a common goal than it is about offloading human tasks to automated processes.

    As a result, DevOps is not nearly as black and white as those zealots might have you believe. For example, the “continuous” in CI/CD doesn’t mean “all the time” or even “as fast as possible.” In practice, continuous means “as fast as practical given the particular business requirements that apply in this situation.”

    Sure, Amazon may deploy several times a minute and be able to easily integrate compliance and governance into their technologies, but that doesn’t mean you can, or even that you must. After all, enterprises have a very different business and technology context from web-scale, born in the cloud companies such as Amazon.

    You may simply have different requirements–and even more importantly, different software will likely have different deployment cadences, even within the same organization.

    These principles of flexibility and human-centeredness apply beyond deployment to all aspects of change–and thus, they apply to the change management and release management practices of ITIL more broadly. 

    Sometimes you need a CAB, and sometimes you don’t. Sometimes you need to change windows, and sometimes you don’t. Sometimes fully automated quality checks are sufficient for giving the green light for deployment, and sometimes they’re not–and so on.

    And perhaps most importantly, sometimes humans should make approval decisions, and sometimes those decisions should be automated. 

    The more you can automate these approvals, the faster your deployment will be. But if you are able to identify those changes that require human approval and route those accordingly, then you’ll have optimized how you handle change in a way that pulls together DevOps and ITIL practices.

    Bringing ITIL and DevOps Together in Practice

    Traditionally, ITSM products are ticketing systems. When some incident occurs or someone calls for a change, they must create a ticket that then kicks off a set of manual tasks, often by different people.

    DevOps can bring automation to this traditional ITSM context. “One of the modern miracles of DevOps principles and practices is that many parts of ITIL processes can now be automated, solving significant challenges, especially in the configuration, release and deployment management processes,” Kim explained.

    Such manual workflows are typically well-suited to supporting ITIL practices, but fall short of the speed that DevOps requires. “ServiceNow DevOps leverages the data and metadata it already has on its platform, combined with new information from the DevOps pipeline to create, automate, and manage ITSM changes from within DevOps workflows on an enterprise-wide scale,” said Jainendra.

    In fact, ServiceNow can oversee the full automation of the deployment, integrating with other DevOps tools as necessary to effect the change, eliminating the need for any additional action from a developer. On the flip side, when the policies in place require a human approval of a change, then ServiceNow routes the change to the appropriate person. 

    The Intellyx Take

    Automation, for all its benefits, has never been about taking humans out of the decision-making loop. On the contrary, automation takes low-value tasks off of people’s plates, allowing them to spend more time and effort on more valuable activities. Decision-making is often among those high-value activities.

    That being said, automation is getting better at making decisions, both because of the breadth of machine-understandable policy representations and other configuration metadata, as well as the increasing prevalence and sophistication of machine learning.

    For DevOps professionals, it’s essential to understand this increasing level of sophistication for the automation available in today’s tools. The choice between automated or manual is no longer a black-and-white affair. Instead, there are different levels of capability for automated decision-making depending upon the situation at hand.

    — Jason Bloomberg