devopsy.kinsta.cloud

Author: Nik Koutsoukos

  • Digital Experience and the Future of Observability

    Digital Experience and the Future of Observability

    Over the past two years, there’s been significant hype around observability in the IT operations and DevOps world. As shorthand for providing better insights, observability has seen a rise in lockstep with distributed architectures, cloud-native applications and the increasingly complex IT environments associated with them.

    As we round out 2022 and move into the new year, there’s evidence observability is gaining mainstream adoption by IT operations and has begun to take off, with the market reaching a better understanding of what observability is and why it matters. This broader adoption offers a moment to reflect on the impact of observability so far and how it will continue to evolve in the year ahead.

    What is Observability?

    Though there’s ongoing debate in the industry about what observability actually means and what an observability platform should include, most people in the industry will argue observability is the evolution of monitoring and promises to offer more complete answers than standalone monitoring. Gartner, for example, first asserted in 2020 observability is “the evolution of monitoring.” But the two aren’t mutually exclusive, and observability and monitoring will continue to coexist and fill distinct needs for different teams based on what information they need about their environments and how they need to act on it.

    So far, observability has had the most considerable impact on application performance and digital experience monitoring.

    The days of monolithic applications are over. Instead, modern applications are highly distributed, are built natively in the cloud using container technology, are organized around microservices and leverage databases in multiple cloud environments. These increasingly distributed software stacks have introduced enormous flexibility for developers but huge complexity for operations teams, making it harder to deliver an optimal experience to the end user. The DevOps teams tasked with building and managing custom web applications are under pressure, and the result is, as one recent article so concisely put it, “Devs don’t want to do ops.”

    Observability has provided a welcome respite for DevOps teams who need intelligent visibility into their environments to determine how applications are performing. This has made application performance a key focus for many observability providers.

    An Intolerance of Bad Digital Experience

    Ensuring external customer-facing applications are performing optimally is the key driver behind this. Digital businesses with external applications core to the company’s success need to ensure those applications perform optimally—customers are fickle. They’re intolerant of bad digital experiences, and alternatives are plentiful. Any e-commerce vendor knows no website equals no sales.

    Traditional application performance monitoring (APM) tools look at telemetry to flag application issues or outages needing to be addressed. But understanding the health of your applications from this narrow standpoint isn’t the same as understanding how the applications perform from a user perspective. In other words, if you aren’t considering how applications are performing (or not!) for customers, employees and other users, you’re not seeing the forest for the trees. Observability helps solve this by elevating APM into a more holistic digital experience approach by adding visibility into how the end user engages with your application.

    By providing a view into how an application is performing with the context of what’s happening throughout the entire stack—including networks, databases, systems, and more—and how issues are impacting the user experience, DevOps teams can shed light on previous blind spots.

    Elevating the importance of digital experience through observability enables teams to understand and respond to issues more efficiently and provides critical data to help DevOps teams further improve their apps beyond just debugging and issue remediation. By understanding how users engage with and experience the application, you can gather critical data to improve or simplify the digital user experience. In our e-commerce example above, this could mean fewer clicks or an easier checkout experience (better digital experience = more sales).

    Cloud migration efforts, coupled with ongoing hybrid and remote work and understaffed tech teams, make prioritizing what matters most even more important. Focusing on the experience of the end user—but with end-to-end visibility across the environment—allows stretched DevOps teams “do more with less.”

    2023 Will be the Year of Digital Experience Observability

    An observability approach prioritizing digital experience is the first step toward building a complete customer-centric environment. This eliminates the potential risks of missing issues directly impacting application performance and creates an enhanced, complete experience for every user. This will make 2023 the year of digital experience observability.

  • Observability Matters. But Are You Observing the Right Things? 

    Observability Matters. But Are You Observing the Right Things? 

    It’s like clockwork. Every few months, a new buzzword descends on the tech industry, and soon it’s on everyone’s lips. Take your pick from the graveyard of buzzwords past: “Big data,” “internet of everything,” “hacktivism,” “gamification.” It’s not that these terms are meaningless—just that they go from relatively obscure to ubiquitous seemingly overnight. We can add a new one to the list in the digital monitoring space: “Observability.”

    Unlike some trendy buzzwords, observability captures something that’s actually quite important about how we monitor digital systems and how current approaches differ from those of the past. So, it’s a good and useful concept. But where I take issue is with the way many analysts, technologists and vendors seem to use it. Whether or not they mean to, they often paint a far too narrow a picture of what observability is or ought to be, and that linguistic oversight can have real-world consequences. If enterprises invest in observability without understanding what, exactly, they’re observing—and what they’re not—they risk leaving themselves vulnerable to some serious blind spots. 

    What is Observability and Why Does it Matter?

    Technically, observability is an approach with the goal of understanding how a system is doing. The distinction lies in what the system is and how it is being observed. The monitoring tools of yesterday typically focused on one or more domains to identify typical and known problems. Usually, that entailed collecting metrics directly from those domains and measuring them against predefined thresholds. For example, storage levels shouldn’t exceed a certain threshold; if they do, send me an alert so I can investigate what’s happening. 

    On the other hand, observability takes a more holistic approach, examining multiple outputs of the system to infer system health. For example, suppose you were trying to gauge a car’s condition. In that case, traditional monitoring might track engine temperature or idle speed RPMs, while observability would examine things like power output, engine leaks, transmission noise, emissions and fuel efficiency to infer the car’s overall health. 

    This output or outcome-focused approach to managing performance maps well to today’s complex and distributed digital systems. Even if you wanted to, you couldn’t pull real-time metrics from every relevant component of a system when those components consist of thousands of microservices running in any number of physical servers across multiple clouds. However, the bigger problem with more antiquated monitoring models is that they only capture the specific metrics they’re designed to track. If you’re encountering a problem you’ve never seen before, your monitoring dashboard might look perfectly healthy while actual system performance is terrible. 

    Observability and APM 

    So then, observability should give you everything you need to track the health of a modern digital business, right? Well, not so fast. Because for today, people seem to use “system observability” interchangeably with “application performance monitoring,” or APM. And they are not at all the same. 

    Yes, you want an observability-based approach to tracking your application environment, and modern APM tools provide a great way to do it. But knowing the health of your application environment is not the same as knowing the health of your overall digital systems as experienced by users. And at the end of the day, if you’re not gauging performance from your users’ perspective—whether that’s customers, employees or other devices relying on your systems—then what’s the point? It’s unfortunately not uncommon to forget that the most important key performance indicator (KPI) of any digital system is the quality of its intended output or outcome. In the case of any application, the most important KPI is the user experience.

    The outputs that APM tools track do contribute to the user experience—and you want to be monitoring them. But so do many, many other things. There’s the user access itself (browsers, virtual or physical user devices loading the application). There are local, regional and global networks connecting those users to your application environment. And then, of course, there are the dozens, sometimes hundreds of third-party digital services running between and among users and applications. Problems in any link of that digital chain can seriously impact the user experience—even if your application environment itself looks healthy. 

    Observing End-to-End, from User to App and Everything in Between 

    To get the full picture of the health and performance of your digital assets, you should be tracking outputs from across the end-to-end digital service chain. That includes telemetry (logs, infra metrics, traces) from your application hosting environment. It should include actively and passively observing at the point of consumption—that is, outputs from customer browsers and worker devices. It must include active and passive monitoring of network connectivity. And it should include continuous active monitoring of all the disparate services—APIs, content delivery networks (CDNs), DNS services, cloud-based security services and every other cloud and web service that contributes to what the user is experiencing. (And by the way, you should look for independent, objective telemetry whenever possible rather than taking what your various service providers report at face value.) 

    When you approach observability from this outcome-based, holistic perspective, you can capture a real and truthful picture of the performance of your full digital business, not just pieces of it. Just as important, when you do identify a problem, you’re in a much better position to do something about it. Because now you have outputs to measure from all the many systems and services that contribute to the user’s application experience instead of just a subset of them, which means it’s much easier to zero in on which piece of the puzzle isn’t behaving the way it should. 

    A Smarter Approach to Observability

    So, should you take all the hype about observability seriously? Yes. As tech buzzwords go, you could do a lot worse. This one actually means something important. But make sure you’re keeping the big picture in mind as you do. 

    Observability that ignores the outcome begins and ends with the application environment and is nowhere near sufficient. We need to expand our sense of observability, begin with the user experience and encompass every link in the chain of the end-to-end digital business. When we do, we create a much more holistic, useful and actionable picture of the health of our digital systems. And we can make sure we’re continually monitoring digital performance in the way that matters most: As experienced by users.