devopsy.kinsta.cloud

Author: Leo Vasiliou

  • The Fallacy of Continuous Integration, Delivery and Testing

    The Fallacy of Continuous Integration, Delivery and Testing

    We know that continuous integration and continuous delivery (CI/CD) have become a DevOps best practice. And many have learned that by adding continuous testing (CT), they can create a virtuous loop, ensuring perpetual code quality and security. They’re not wrong.  Yes, testing continuously is good practice, but some incorrectly equate the concept of continuous with monitoring, thinking they’re able to see everything that could eventually affect a customer’s experience (CX) with their software application. That’s a fallacy. Many of the most glaring blind spots that wreak havoc on CX exist outside the environment that CI/CD/CT is able to “see.” CI/CD/CT, even when supported by application performance monitoring (APM), doesn’t and cannot monitor for and alert to blind spots across the internet stack. And there are many. Only internet performance monitoring (IPM) can take CI/CD/CT as input streams and shift to continuous resilience as a measurable outcome.

    How CI/CD/CT Typically Works

    As customer requirements evolve, software development teams write code that triggers a new software build. In the continuous model, each new build moves to a runtime environment for integration and quality assurance and, ultimately, is deployed to end users across public or hybrid clouds. This is an application-specific view of the world.

    Since quality assurance is part of the typical continuous process, monitoring is an assumed benefit or outcome. This assumption might be because many now complement their CI/CD/CT processes with APM. What is achieved with APM in this approach is testing, not monitoring. While testing provides a better picture of performance at the application level, it doesn’t provide full visibility into the blind spots that exist between the application layer and the end user.

    APM Isn’t Enough

    While APM tools focus on code—including everything that negatively affects an application such as database wait times and inefficient code—IPM focuses on the internet, which has become your network. IPM monitors what impacts your customers, workforce or application (API) experiences across the internet.

    While APM and IPM may seem similar, they are nothing alike. They may share common tools, such as using synthetic agents, but the difference is in what they monitor and how. In this age of highly distributed applications, that difference means everything.

    Where APM can “simulate” what a user may see from cloud location to cloud location, which can confirm application performance, this simulation isn’t true or comprehensive monitoring because it is blind to network-related problems with end-user experience, which are vast. Customers and employees each have a unique internet performance fingerprint and access applications from myriad devices across multiple locations and network infrastructures, not through the individual cloud locations that APM synthetics simulate.

    IPM approaches synthetic testing more holistically, replicating multiple user journeys across a much more complex Internet infrastructure to proactively flag performance issues before they impact end users. This true monitoring takes place 24/7, not as point-in-time tests.

    Forensics vs. Monitoring

    With APM and similar observability solutions that embrace real user monitoring (RUM), it’s important to focus on what monitoring really means. APM passively collects data that is a proxy for user experience, enabling postmortem assessment of causes and possible fixes, but after the fact. Observability looks at logs, metrics and traces and provides an assessment of whether users are having a good experience or why they did not. Neither solution by itself will prevent poor experience, and many fixes won’t take place until a postmortem analysis feeds the next iteration. The poor experience has already done its damage, and the costs are already sunk.

    With thousands of potential blind spots across the globe that can slow or disrupt applications, from ISPs and wireless carriers to CDNs, it’s not hard to grasp the increasing and differentiated importance of IPM. Where the combination of CI/CD/CT with APM can proactively prevent deployment of poor code that would affect user experience, only IPM is continuously replicating – even more valuable than monitoring–millions of experiences across a dynamic multi-node network where the unexpected is now expected.

    Internet Resilience Vs. Retrospection

    Businesses today understand the importance of resilience: It’s a board-level concept. Any business that has lost market share and market cap during sustained downtime or poor user experience understands what’s at stake: Millions if not billions of dollars. IPM is squarely focused on “internet resilience,” and its pillars are availability, reachability, performance and reliability. For businesses reliant on the cloud and employers operating with highly distributed workforces, retrospection alone is a risky proposition. Now that the internet is the new corporate network, internet resilience is a new organizational mantra.

    The Takeaway

    Whether your organization embraces CI/CD/CT already or is rethinking its approach to DevOps, this article should give you pause. Your job–perhaps as part of a larger team–is to catch performance issues and potential disruptions with your application before client impact is realized. Without IPM, only part of that job is being done.

    Why is this so important? Even conservative estimates put the cost of an outage at $6,700 per minute (Gartner) and the “cost of slow,” where something causes an application to perform poorly and it goes unnoticed, is likely significantly higher because it happens all the time. The fallacy isn’t that there’s value in CI/CD/CT–there is. The fallacy is that you’re in control of the entire CX of your internet-deployed products without monitoring the internet stack and ensuring internet resilience. Only by adding IPM can you have this assurance.

  • SREs Say AIOps Doesn’t Live Up to the Hype

    SREs Say AIOps Doesn’t Live Up to the Hype

    What can you expect when investing in artificial intelligence for IT operations (AIOps)? Real-time visibility across huge volumes of information. Lightning-fast event correlation and anomaly detection. Automated remediation and self-healing, without Ops personnel having to lift a finger.

    It all sounds amazing—at least, if you believe the marketing hype. But do those kinds of AIOps benefits actually translate to the real world? According to a recent survey of nearly 300 site reliability engineers (SREs), not so much.

    Among SREs surveyed—including engineers for some of the world’s largest, most innovative digital companies—just 12% use AIOps as a day-to-day part of their monitoring toolkit. Nearly 40% never use it at all.

    Why aren’t current AIOps solutions living up to the hype? And what can this gap tell us about the state of IT monitoring in general? Let’s take a closer look.

    Sprawling Complexity

    In theory, AIOps should be a perfect fit for SREs. These are, after all, the people with one foot in the world of operations, one in development, whose job is to make distributed software systems ultra-scalable and reliable. A solution that can turn mountains of data about those systems into actionable insights should be exactly what SREs need. And, in fact, 53% of SREs cite lack of unified visibility across the stack as their biggest cloud application monitoring challenge.

    And yet, if we gauge by utilization, today’s AIOps tools don’t seem to be delivering on that promise. Asked about the value of AIOps, just 7.5% of SREs, on average, view AIOps as “high-value”—even less than those SREs (9%) who perceive its value as low.

    Why is AIOps so slow to catch on? Ultimately, the barriers facing these tools are the same as those facing human engineers: massive and growing complexity in IT environments. As digital products become more dependent on third-party cloud services, as the number of things businesses want to track grows (from infrastructure to application to experience), the sheer volume, velocity and variety of monitoring data has exploded.

    Diverse Data Sources

    Compounding the problem, enterprises increasingly rely on multiple “same-service” providers for IT services. That is, they use multiple cloud providers, multiple DNS providers, multiple API providers, etc. There are sound business reasons for doing so, such as adding resiliency and drawing on different vendors’ strengths in different areas. But even when two providers are doing basically the same thing, they use different interfaces and instrumentation, and their data sources often employ different metrics, data structures, and taxonomies.

    Whether you’re asking a human being or an AI-driven tool to solve this problem, this heterogeneity makes it extremely difficult to visualize the complete picture across the infrastructure. It also creates gray areas around how best to take advantage of each vendor’s different rules and toolsets. All of which makes automating anything beyond the most superficial operations tasks hugely challenging, if it’s possible at all.

    Optimizing AIOps

    AIOps really is the solution to this problem—or at least, an important part of the solution. Machine learning is, after all, tailor-made to analyze large volumes of observational and engagement data and react in real-time, as well as to continually identify opportunities for improvement. But before we can get there, we need to change the way we think about AIOps.

    Don’t try to do everything at once.

    It would be great if we could deploy AIOps, let it learn about our environment and watch it magically start remediating issues on its own. But we’re not there yet. (And if a vendor tries to convince you we are, you should definitely be skeptical.) If we break down our expectations into smaller, more manageable chunks, however, AIOps can accomplish quite a lot.

    Event correlation, topology analytics and smart alerting, for example, are all areas where AIOps can provide significant value right now. For these more limited objectives, AIOps can absolutely reduce tedious manual effort for human operators, reduce mean time to repair, improve visibility and more, and digital businesses should be leaning on AIOps to do that.

    Invest in AI and ML training.

    Part of the reason adoption of AIOps tools remains sluggish stems from the fact that many teams still just don’t fully understand how they can use them. The more SREs are exposed to AI/ML training, the more opportunities they will identify for concrete, meaningful applications of AIOps. For example, only 27% of SREs said, “Experimenting or receiving training to expand knowledge or skills” was a major activity as part of their job.

    Embrace “Platform Ops.”

    The biggest step we can take is to rethink how we use AIOps within an organization. Today, AIOps tools—like SRE resources themselves—are often dedicated to a specific digital product or service. As a result, enterprises end up trying to apply their AIOps tools, and the SREs using them, to support everything associated with that product (DNS, API, CDN, cloud monitoring, etc). As IT environments continue to get even more complex, though, businesses will likely find that shifting to shared platform tools and teams yields better results.

    Instead of trying to apply AIOps across every part of a digital service, imagine applying it to one part of that service (for instance, focusing on just DNS, or just APIs). In this way, the AIOps tools, as well as the teams using them, gain in-depth expertise across all the different interfaces and instruments used for that function and get continually better at normalizing multi-provider data sources across them. At that point, this platform-specific AIOps capability can then be scaled and reused across the company—providing more advanced DNS AIOps (or API, or CDN, etc.) for multiple digital products and teams.

    Now, you’ve got an advanced, highly specialized toolset for optimizing an important platform function, regardless of which digital product happens to be using it. You’ve narrowed the scope of how you’re using a given AIOps implementation while going much deeper into that function than a more generic, broad-based implementation could achieve. All of a sudden, even more advanced automation and self-healing becomes possible. And the loftiest marketing claims of AIOps start to look less like hype and more like viable real-world outcomes for your business.