Tag: Logz.io

  • Logz.io Unveils Observability Platform Designed First for AI Agents

    Logz.io Unveils Observability Platform Designed First for AI Agents

    Logz.io today launched an edition of its observability platform that is intended to be primarily used by artificial intelligence (AI) agents rather than DevOps engineers.

    Company CEO Tomer Levy said that as DevOps continues to evolve it’s become apparent that most tasks pertaining to observability will soon be assigned to AI agents. This edition of the Open 360 observability platform exposes machine-readable schemas alongside every panel, allowing AI agents to autonomously create charts, interpret anomalies and modify entire dashboards.

    Rather than simply adding a chat bot interface to an existing observability platform, Logz.io is moving to make it possible for DevOps engineers and AI agents to collaboratively observe complex IT environments using a common set of dashboards, said Levy.

    That approach still produces insights via a dashboard that can still be read by humans, but the assumption is that AI agents will be the primary end user of those dashboards, he added.

    Tasks assigned to those AI agents include executing custom playbooks created by DevOps teams, reviews of new application deployments, cost analysis and security assessments. AI agents can also automatically generate visualizations based on natural language requests, interpret and analyze anomalies in real time and interact with other agents to automate more complex tasks.

    Logz.io has already tested this AI agent-first approach with more than 200 IT organizations that have deployed more than 2,000 Logz.io AI Agents, resulting in saving a total of approximately 25,000 engineering hours this month alone—equivalent to 12 full-time SRE positions. At the current adoption rate, Logz.io projects customers will recover roughly 300,000 hours of engineering time over the next year.

    It’s not clear how extensively AI agents will change the way DevOps workflows are managed, but it is apparent that with the rise of AI coding tools, the number of applications being deployed by any organization is about to significantly increase. Most existing DevOps teams are not going to be able to keep pace with that rate of application development and deployment without being augmented in some way by AI agents.

    The challenge then becomes determining which tasks are best assigned to an AI agent versus those that still require the expertise of a DevOps engineer. Over time, however, DevOps engineers will find themselves supervising a small army of AI agents that have been trained to perform a range of specific tasks.

    Ultimately, AI agents should make applications both more resilient and secure as anomalous behavior is identified sooner. Armed with those insights, it then becomes feasible for application developers that are similarly augmented by AI agents to troubleshoot applications in a way that results in more issues being resolved long before they have any meaningful impact on the organization.

    In fact, AI agents should make it possible for more organizations to benefit from observability by proactively addressing application issues rather than simply continuing to monitor a set of pre-defined metrics that provide limited actionable intelligence. That, of course, might take some time for IT professionals to accept and fully trust, but in time, many of the tasks that conspire to make managing DevOps workflows more tedious will, like it or not, be increasingly performed by AI agents.

  • Survey Surfaces Lack of Significant Observability Progress

    Survey Surfaces Lack of Significant Observability Progress

    A global survey of 500 IT professionals suggests organizations are not making a lot of progress in their ability to truly observe application environments, especially as they become more complex with each passing day.

    The survey, conducted by Logz.io, a provider of an observability platform, found only one in 10 respondents reported they have full observability into their application environments.

    Logz.io CTO Asaf Yigal said while more DevOps teams than ever are collecting logs, metrics and traces, most of them have yet to determine how to operationalize all the data being collected. As data is ingested, it doesn’t just need to be stored; it also needs to be correlated to the various services that make up an application, he noted.

    The more organizations deploy cloud-native applications in production environments the more pressing this issue becomes. All the microservices that make up those applications are now generating a massive amount of telemetry data that results in more alerts than ever being generated.

    Not surprisingly, the biggest challenges organizations encounter when managing Kubernetes clusters in production environments are monitoring/troubleshooting (40%), followed closely by security (37%) and networking (33%).

    Many organizations simply don’t have the skills required to manage cloud-native applications. Nearly half of respondents (48%) specifically cited a lack of knowledge as the biggest challenge they encountered when trying to observe these types of applications. From an IT management perspective, most microservices all use the same basic template, so it’s difficult for DevOps teams to identify which microservices are likely to have the biggest impact on service level objectives (SLOs) and service level agreements (SLAs) in the event of a disruption, noted Yigal.

    In the absence of the ability to determine the actual root cause of an issue, alert storms conspire to increase fatigue—and that eventually results in higher levels of burnout across the DevOps team, noted Yigal. In fact, 82% of respondents said their mean-time-to-resolution (MTTR) during production incidents was more than an hour.

    The survey also noted that more than half of respondents (52%) worked for organizations that are simultaneously trying to rein in monitoring costs. More than three-quarters (76%) of respondents also noted that open source OpenTelemetry (OTEL) or OTEL-centric tooling was at least somewhat important to their overall observability strategy.

    In addition, 87% of respondents said their organization is already using some form of platform engineering to manage DevOps workflows at scale.

    It’s not clear how quickly organizations are adopting observability tools and platforms, but too many DevOps teams lack the visibility needed to identify the root cause of an issue. As a result, variations of the same problems often keep manifesting themselves because previous remediation efforts simply didn’t go deep enough to resolve the core issue.

    Of course, there may come a day when machine learning algorithms, along with other forms of artificial intelligence (AI), will make it simpler to surface these issues. The challenge, in the meantime, is laying down the observability foundation today to provide access to the data that will be required to train those AI models.

  • Logz.io Leverages AI to Identify Anomalies in Real-Time

    Logz.io Leverages AI to Identify Anomalies in Real-Time

    Logz.io this week revealed it has added a real-time anomaly detection capability to its observability platform that makes it simpler to correlate the impact IT events will have on business processes.

    This capability is based on artificial intelligence (AI) models trained using historical data collected via the Logz.io Open 360 platform.

    Logz.io CTO Asaf Yigal said this addition to the Open 360 platform enables DevOps teams to employ machine learning algorithms to identify anomalies in mission-critical processes that warrant further investigation. Armed with insights that go beyond simple IT infrastructure consumption metrics, it becomes possible to significantly reduce mean time to resolution, he added.

    Logz.io has been embedding AI models within its platform since last year. Still, this latest capability will enable organizations to further bridge a divide between IT and the business that has long plagued many organizations, noted Yigal.

    The ultimate goal is to make it simpler for DevOps teams to instantly identify any anomalous application behavior that represents a risk to the business, including cybersecurity events, he added.

    The AI capabilities that Logz.io has been steadily adding to its observability platforms are largely transparent to DevOps teams, said Yigal. Instead, Logz.io is embedding multiple types of AI models into its platform that can be invoked in much the same way any other capability is provided, he added. DevOps teams are not necessarily going to need to build their own models to take advantage of AI to observe application behavior at scale, noted Yigal.

    The goal is to surface those capabilities consistently at a level of cost organizations can afford, added Yigal.

    It’s not likely AI will replace the need for DevOps teams any time soon, said Yigal. After all, there is a reason AI models are referred to as co-pilots versus an automatic pilot, he noted.

    It’s not clear to what degree organizations today can correlate the impact an IT event is going to have on the business. But as organizations continue to realize how dependent they are on applications to drive digital processes, the need to bridge the historic divide between IT and the rest of the business has become more urgent. In fact, many of the product and brand champions that organizations now rely on to manage a business unit often have a background in DevOps.

    Of course, business professionals that are steeped in IT are still relatively rare but as advances in AI continue to make it simpler to understand how complex application environments function the number of users of an observability platform will naturally expand beyond IT. The challenge is presenting each stakeholder with a view into the IT environment that meets their specific requirements.

    It may take a yet before the divide between IT and the rest of the business is permanently bridged but with each passing year it gets narrower. The challenge and the opportunity now is making sure everyone on both sides of that divide understands the art of the possible today that only a few short years ago would have seemed impossible to achieve.

  • Logz.io Taps AI to Surface Incident Response Recommendations

    Logz.io Taps AI to Surface Incident Response Recommendations

    Logz.io this week added a supervised machine learning capability to its observability platform that reduces mean-time-to-remediation by surfacing recommendations for resolving incidents.

    Asaf Yigal, vice president of product for Logz.io, said the Alert Recommendation capability added to the Logz.io Open360 platform uses artificial intelligence (AI) to model the steps a DevOps team needs to complete to resolve an incident.

    The goal is to reduce the amount of time required to resolve incidents at a time when IT environments are becoming increasingly complex, he added.

    In fact, a recent Logz.io survey found 75% of respondents said it currently takes them hours to resolve production issues, with only 14% satisfied with their current mean-time-to-resolution (MTTR). A total of 41% specifically cited monitoring and observability of Kubernetes environments as a primary challenge.

    Alert Recommendation is the latest in a series of investments in AI that Logz.io has made. Previously, Logz.io integrated the ChatGPT generative artificial intelligence (AI) platform to surface links to related information and best practices for resolving IT issues.

    In general, AI tools should make it possible to manage IT at a level of scale that eliminates many of the low-level data engineering and analytics tasks that previously required manual effort from a DevOps engineering team. If, for example, the observability platform is generating recommendations to address issues, there may be less of a need to create runbooks that DevOps teams typically create to address a wide range of known issues.

    Along with making it simpler to discern the root cause of an issue, AI also makes it possible for less experienced members of a DevOps team to resolve an issue using guidance generated by the observability platform, noted Yidal.

    In effect, AI technologies are reducing the cognitive load required to be an effective member of a DevOps team, he added.

    One way or another, it’s not so much a question of whether AI will be applied to DevOps as much as it is to the degree. Many of the manual tasks that often conspire to create DevOps bottlenecks should be significantly reduced in the months ahead as more advances are made. The challenge now is determining how best to reallocate DevOps expertise in anticipation of those advances.

    Of course, there’s always going to be some sense of trepidation when it comes to AI. However, many of the tasks that are about to become automated tend to be tedious. Many DevOps professionals would just as soon see those tasks become automated in the expectation that more time will become available to take on more complex challenges.

    Regardless of the motivation, the way IT is managed is about to change. There may be plenty of instances where AI does not live up to its initial hype, but as AI models are exposed to more data, they will become more accurate. That doesn’t mean, however, there won’t always be a need for a DevOps engineer to ensure those algorithms are behaving as expected.

  • Logz.io Integrates ChatGPT With Observability Platform

    Logz.io Integrates ChatGPT With Observability Platform

    Logz.io has integrated the ChatGPT generative artificial intelligence (AI) platform with its Open 360 observability platform to extend its automated recommendation capabilities to reduce mean-time-to-remediation.

    ChatGPT, developed by OpenAI, is based on a large language model that Logz.io uses to enrich crowdsourced data. That approach enables ChatGPT to surface links to related information and best practices for resolving IT issues.

    Logz.io CTO Asaf Yigal said the company has integrated ChatGPT with its existing AI engine, dubbed Cognitive Insights, that applies machine learning algorithms to identify anomalies indicative of potentially critical IT issues. The company expects to be able to add other generative AI capabilities as additional advances are made, he added.

    The Logz.io Open 360 platform is based on open source OpenSearch, Prometheus and Jaeger software that the company has infused with machine learning algorithms. The integration with ChatGPT is enabled via an application programming interface (API) that OpenAI has provided.

    The overall goal is to employ AI to surface the most relevant issues a DevOps team needs to focus on without having to search huge volume of logs, tracing and metrics data, said Yigal. Generative AI platforms solve a fundamental cognitive load problem created today as DevOps teams collect more data, noted Yial. The more data a DevOps team collects, the harder it becomes to determine what’s going on in an IT environment, said Yigal. He added that generative AI platforms make it possible to analyze massive amounts of data at a scale that isn’t otherwise achievable.

    AI tools should make it possible to manage IT at a level of scale that eliminates many of the low-level data engineering and analytics tasks that previously required manual effort from a DevOps engineering team. It’s not clear how many other DevOps tools and platforms will be invoking generative AI capabilities in the months ahead, but DevOps teams should assume that large language models will soon be augmenting DevOps teams in ways that previously would have been thought unlikely.

    For example, discovering vulnerabilities in software running in production environments is about to become trivial as artificial intelligence (AI) platforms such as ChatGPT evolve. A generative AI platform can identify code vulnerabilities in much the same way as a traditional scanning tool. An AI platform, however, could theoretically monitor code repositories and scan them for vulnerabilities as updates and commits are being made in real-time. DevOps teams should be able to use generative AI platforms to discover and remediate these issues before code is deployed in a production environment.

    It’s just as probable bad actors will use the same platform to achieve the same goal, so from that perspective, many DevOps teams will now find themselves in an AI arms race to improve DevSecOps workflows.

    There are still lots of issues to be worked out in terms of the overall accuracy of generative AI platforms, but they will only become more intelligent as processors continue to advance. The issue now will be determining precisely where the man-machine interface for DevOps teams is today and in the future.

  • Logz.io Unveils Managed Open 360 Observability Platform

    Logz.io Unveils Managed Open 360 Observability Platform

    At the AWS re:Invent 2022 conference, Logz.io launched an Open 360 platform that combines multiple open source technologies to provide observability across both existing monolithic and emerging cloud-native application environments.

    Existing Logz.io offerings included in the Open 360 platform are Logz.io Kubernetes 360, a managed observability platform based on open source tools such as OpenSearch analytics software, Prometheus monitoring tools and Jaeger distributed tracing agents.

    In addition, Open 360 includes Logz.io Telemetry Collector agent software, Logz.io Data Optimization Hub dashboard, Logz.io LogMetrics Index for converting log data into metrics and the Logz.io Trace Sampling Wizard tool for configuring open source OpenTelemetry Collector agent software.

    Logz.io CEO Tomer Levy said rather than requiring organizations to combine a set of disjointed open software tools to create an observability platform, instances of open source tools have been curated by Logz.io. The goal is to streamline the number of tools and platforms that DevOps teams will need to observe multiple classes of applications, he noted.

    It’s still early days as far as the adoption of observability platforms is concerned, but it’s apparent that as application environments become more complex the monitoring tools DevOps teams rely on today will need to evolve. Observability platforms, in general, promise to unify logs, metrics and traces in a way that makes it simpler to launch queries to identify the root cause of an issue rather than simply tracking a set of pre-defined metrics.

    The rate at which DevOps teams will embrace observability will naturally vary, but the choice many DevOps teams are trying to navigate is how much to rely on proprietary observability platforms rather than using open source software to construct their own. Logz.io is making a case for finding a middle ground between those two extremes by relying on a managed service based on open source tools.

    Observability has always been a core DevOps tenet, but achieving and maintaining it is challenging. Most DevOps teams today aspire to maintain some level of continuous monitoring. However, as it becomes easier and less costly to instrument applications, interest is rising in observability platforms that simplify the investigation of anomalies indicative of an issue that could disrupt a distributed application environment.

    More challenging still, DevOps teams are also now managing a much wider range of application types as cloud-native applications based on microservices deployed on Kubernetes clusters are deployed alongside legacy monolithic applications running in the cloud and on-premises IT environments. The Logz.io approach enables DevOps teams to manage those applications at a lower total cost, noted Levy.

    Regardless of the approach to observability, the increasing complexity of application environments has become a major challenge for organizations that are increasingly sensitive to costs. Many of them can’t afford to hire additional DevOps engineers. Instead, the focus is on increasing the productivity of existing DevOps teams using a new generation of observability platforms that should cost less to acquire and deploy than to expand existing teams of DevOps engineers. That may not necessarily eliminate the need to hire additional DevOps engineers, but it should ensure organizations are getting the most out of the DevOps teams they already have.

  • Open Standards Are Key For Realizing Observability

    Open Standards Are Key For Realizing Observability

    Observability has quickly become a major focus of enterprise DevOps. Yet tool sprawl and complexity can hold some observability initiatives back. The 2022 CNCF Microsurvey on observability trends found that 23% of organizations use between 10 and 15 tools for monitoring, metrics and gathering logging and tracing data. A full 50% also noted that engineers and teams using multiple tools presented a top challenge to observability.

    I recently met with Dotan Horovits, principal developer advocate for Logz.io, to get his take on improving how modern development teams handle observability. According to Horovits, open standards and open source projects are the principal drivers to realizing interoperable observability industry-wide. Below, we’ll consider the issues facing observability and introduce OpenTelemetry, a CNCF project quickly gaining traction to unify telemetry data from various sources.

    Open Standards For The Win

    Although proprietary monitoring agents have a lot of intelligence, they inherently use tight coupling between telemetry generation and the backend storage analytics, said Horovits. This can cause vendor lock-in and lead to formatting variations in telemetry data between toolsets. “The open standards have the power to converge our industry to prevent vendor lock-in, even the playing field, and allow us to get any telemetry for any source,” said Horovits.

    Without an open standard, supporting the evolving suite of technologies at the heart of modern polyglot software stacks is challenging. Organizations constantly add to their tech stack, from SQL to NoSQL, Kafka, API gateways and serverless cloud services, and these modules produce varying data formatted in unique styles. What’s more, companies are typically using more than one cloud vendor, further necessitating the need for a decoupled method to represent multi-cloud metrics. “As soon as your IT stack becomes this complex, there is a need to aggregate all telemetry into one unified observability standard,” explained Horovits.

    Introducing OpenTelemetry

    OpenTelemetry is described as a “high-quality, ubiquitous and portable telemetry to enable effective observability.” Using OpenTelemetry (sometimes abbreviated as OTel), engineers can generate, collect and export metrics, logs and traces. OpenTelemetry works with many popular libraries and is completely vendor-neutral and supported by a rising number of tools in the space. OpenTelemetry was accepted to CNCF on May 17, 2019, and is an incubating project.

    The OpenTelemetry registry boasts hundreds of libraries and plugins to export and transform telemetry data from many different environments, from legacy to contemporary sources. By decoupling the data collection and export process, OpenTelemetry is “one agent to rule them all,” Horovits explained. Of course, he added, vendors who adopt OpenTelemetry may still provide their own logic behind it.

    The Benefits of More Open Telemetry Data

    At the time of writing, live public repository data shows OpenTelemetry is the second-most active CNCF project, behind Kubernetes. So, why is the observability community so excited about OpenTelemetry? Well, for one, the interoperability it affords could equate to efficiency and time savings. “OpenTelemetry gives you the freedom to integrate throughout the stack,” stated Horovits. Part of this is because its open source roots make it a force multiplier. “You can never achieve such vast coverage with closed source as you would with a wide-open community,” Horovits said.

    Arguably, an open telemetry toolset has also become necessary due to the sheer complexity of cloud-native architectures. Most organizations are shifting to a distributed microservices architecture supported by a dynamic, elastic container-based infrastructure. In this world, a single service might span multiple clusters. Therefore, engineers might need to consider clusters, pods, nodes and namespaces when filtering for data.

    In this complex reality, there are many dimensions, multiplying what needs to be monitored. And in general, traditional monitoring systems become ineffective at this level. Tool sprawl and lack of consolidation are problems that arise when trying to cater to many environments simultaneously. Thus, a unified, decoupled observability platform makes a lot of sense.

    Future of OpenTelemetry

    We’ve seen vast adoption from cloud vendors and solution providers around OpenTelemetry. And, the project has been moving swiftly—OpenTelemetry has had a distributed tracing specification released for about a year. The project almost has a release candidate for metrics and one for logs is in the works, said Horovits.

    Beyond that, one feature that Horovits is excited about is continuous profiling, which could help users continuously analyze a function and track usage trends across instances. Another exciting potential development is eBPF. Since it’s language-independent and runs at the kernel level, eBPF could extract observability signals without requiring application-specific hooks or integrations. This could increase support for more environments.

    For more information, you can visit the OpenTelemetry site to peruse the documentation or read up on the feature roadmap for OpenTelemetry. Horovits also gave this helpful presentation on OpenTelemetry at KubeCon 2022.

  • Logz.io Extends Open Source Observability Platform

    Logz.io Extends Open Source Observability Platform

    Logz.io has enhanced its observability platform to include both unified dashboards and support for distributed tracing to monitor service performance enabled by the open source Jaeger agent software being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    At the same time, Logz.io is making available a preview of an anomaly detection capability that leverages machine learning algorithms to build an artificial intelligence (AI) model to monitor data.

    The company has also added support for OpenSearch, a fork of the open source ElasticSearch software project that is no longer available via an Apache license. OpenSearch was created by Amazon Web Services (AWS) and Logz.io as an alternative search engine.

    Finally, Logz.io has added a security event management capability that makes it easier to classify, prioritize and collaborate on mitigation of cyberattacks as they are discovered.

    The Logz.io observability platform is entirely based on open source software. In addition, the company has made contributions to enhance both Jaeger and the OpenTelemetry project, including contributing an OTEL Span Metrics Processor that makes it easier to aggregate metrics.

    Logz.io CEO Tomer Levy said that, in general, it’s still early days as far as observability is concerned. As a concept, observability traces its lineage to linear dynamic systems. Observability at its most basic level measures how well internal states of a system can be inferred from knowledge of its external outputs. The overall goal is to make it easier to query machine data in a way that enables DevOps teams to proactively discover the root cause of issues before they cause further disruption. That’s a major advance over monitoring platforms that typically only provide predefined metrics to identify when a specific platform or application is performing within expectations. The issue, however, is most IT teams really don’t know what questions to ask to take full advantage of an observability platform, said Levy.

    However, as more organizations are exposed to observability platforms, it will become easier to advance both DevOps and DevSecOps as more platforms continue to leverage AI to provide remediation capabilities and automate processes based on the massive amounts of data collected via these platforms, said Levy. There is simply no way the average IT team will be able to manage all the microservices being strewn across a distributed computing environment without the aid of an AI model trained using observability data, he added.

    As DevOps continues to evolve, it’s more than likely that the best DevOps engineers are not going to want to work for organizations that have not invested in some form of observability. The odds of being consistently successful as a DevOps engineer decline sharply as IT environments become more complex.

    On the plus side, it should soon become a lot easier as the overall level of application instrumentation increases to manage those complex environments and to do so at a level of scale that doesn’t require a small army of DevOps engineers.

  • Open Source is the Engine for Observability at Scale – Observability at Scale TV EP 2

    Open Source is the Engine for Observability at Scale – Observability at Scale TV EP 2

    As today’s organizations seek advanced monitoring and observability to ensure the performance of their microservices and cloud native systems, issues of cost, complexity and tool sprawl have emerged as pervasive challenges. Join our hosts Alan Shimel and Mitch Ashley along with panelists Dotan Horovits (Logz) and Constance Caramanolis (Splunk) as they dive into the challenges that DevOps practitioners are frequently forced to compromise on and how they transform modern observability. The video and a transcript of the conversation are below. 

    Alan Shimel: Hey everyone, welcome to Observability At Scale, one of the latest video series or video show series, over here on TechStrong TV part of TechStrong Group. I’m your host, Alan Shimel. And let me just say that Observability At Scale is sponsored by our good friends at Logz.io, it’s L-O-G-Z and many thanks to them for sponsoring this. This is a great subject.  

    I recently in KubeCon about two weeks ago. And I will tell you, it was pretty much all Observability all the time. I did 40 interviews there in two days. And I wouldn’t be surprised if somewhere between a third and 40% of the people I interviewed were about Observability. And many of those being at KubeCon CNCF Cloud Native Con, many of them were about open-source Observability. 

    Whether we’re talking about Prometheus or OpenTelemetry or several of the other Observability-related open-source projects out there. That seems to be the subject of today’s episode: Observability and open source. And I think we’ve got a great panel to discuss this. Let me introduce you to them. First of all, we have Dotan Horovits. Dotan is with Logz. Actually Dotan, you introduce yourself. I don’t want to put words in your mouth. Go ahead.

    Dotan Horovits: Gladly. Glad to be here, Alan. And my name is Dotan Horovits, and I’m a developer advocate at Logz.io. Logz is a cloud-native platform for observability. And the nice thing that I love about Logz.io is that it’s all based on popular open-source tools that we’re going to talk about extensively today like the ELK Stack and Prometheus and OpenTelemetry and Jaeger and so on. So that’s about Logz and myself.

    Alan Shimel: Thank you, Dotan. Next up, we have Constance Caramanolis, and I hope I did that justice. Hey, Constance, how are you? Maybe a little bit about yourself for our audience?

    Constance C.: Good morning. I’m good. Hi, everyone. I’m Constance. I’m a principal software engineer at Splunk. Right now I’m on OpenTelemetry Governance Committee for I guess another five days until elections are over. But I’ve been involved in open source for about four or five years now, both on Envoy OpenTelemetry and on the product with Con community.

    Alan Shimel: Excellent. Thank you, Constance, for coming on today. We appreciate you having you here. And then last but not least, is my co-host of Observability At Scale. He’s also CTO here at TechStrong Group and a principal analyst at TechStrong Research. It’s Mitchell Ashley. Hey, Mitchell, how are you?

    Mitchell Ashley: Good, good, good. Great to be here and very thankful to Dotan and Constance for joining us. I look forward to a great conversation.

    Alan Shimel: Yep. Thank you. So guys, as I started the show off I was out in Los Angeles, I guess it was two weeks ago now for KubeCon Cloud Native Con. And first of all Linux Foundation did an amazing job creating as safe an environment as possible for us to do an in-person event. And it was great to be in-person they got. I don’t want to belabor it but this stuff is good on Zoom, it is better in-person. 

    One day maybe we’ll run this as a live panel from a KubeCon, and that would be just great. But it was all Observability all the time, it seemed. Several things with Observability. Number one, the hairs on the back of my neck stand up when I go to a major show. And there’s something that’s so hot, so hot, that I just feel I can never live up to its promise. And that is what I was afraid with you all. 

    Are we in a overhype cycle around Observability. And then secondly, so much of Observability is based on open-source underpinnings, if you will. Right. There’s a lot of great open-source programs. Maybe that’ll save us from the usual hype cycle that we get here, I don’t know. Anyway, I’m interested in your comments on that, is Observability being overhyped at this point?

    Dotan Horovits: If I may take the first pass on this one. I think that as many trends, and we’ve seen the quite a few of those thought out the years, you have so many vendors trying to piggyback on the trend and leverage that for the relevant offerings that it creates a bit of a blur and confusion about what exactly Observability is about.

    Is it about tooling or methodologies? Is login in its own right for vendors that just provide, let’s say, login of vendors just provide metrics is that Observability? And can you rebrand for Observability yes or no? You have two of these, is that enough? Is signals the whole story about Observability at all? Or is the discussion about Logz metrics traces do justice to Observability?

    I think these are the sorts of conflict and confusion that I see and sense in my conversations with the user community, with customers of ours, with partners of ours. And, again, lucky enough in my position as an evangelist, I use my time to try and clarify and try to separate between the tooling and the methodologies and the essence of things. But definitely, a lot of confusion around that.

    Alan Shimel: A lot of confusion. I did an interview or a panel discussion yesterday with some folks. And one of the things, there was a good line that came out of there was actually around the definition of DevOps, but it’s equally, I think, applicable here, which is, what is Observability? And all of these kinds of things you mentioned, Dotan is, where is the value? 

    Show me where the value is in and I’ll show you what it is, who’s using it, who’s doing it, who’s not. Where is the value? Constance, one might say you were into Observability before it was cool, before it was hyped.

    Constance C.: True. Yeah, right, because that was one of the selling points of Envoy, was making things transparent to the networking to everyone around. And one of the features that made people really love the project was that we would joke, there’s a staff for that. You would send a request, there would be staff for the connection, and everything related to it, and so that made things really easy to debug as a consumer of it. 

    Dotan, you’re definitely right in terms of so much confusion because observably can almost be like the block-chain. And no offense of block-chain, but that’s like the key word that gets thrown around in terms of getting really hip and cool. I think the nice thing about this attention is that it is forcing a lot of people to start thinking more about why they’re doing things.

    There’s a lot of just doing things because you’re either told to or your thought about and now sort of be like, hey, if you’re going to do something, we’ve always asked how do we know at work, but it’s like, oh, let me just write a test. Now it’s a little bit more like how do I systematically know it works by either adding instrumentation so I can consistently monitor it or observe it. 

    I think that is a really good thing that we need to start moving towards, because it’s a little bit more of a, we still check something in, throw it over our shoulders and be like, whoa, good day, everyone. And we’re starting to think a little bit more that sustainability. 

    And I also think there’s a second part too, especially when it comes to these projects that were mentioned earlier, we’re not just thinking about tools anymore, we’re sort of seeing them as products and starting to actually say why this could matter to you, why you should actually adopt them. 

    And that’s something that as engineers, it’s not in our normal toolset in how we think about things, we’re starting to get better at, because we need to start. We’re also selling things for open source, the currencies, commits and contributions and not necessarily money.

    Alan Shimel: I think that’s super important. To the question, is it overhyped? Will it be overhyped? Yes, yes. And whatever the next question is, yes. But I think what Observability has going forward is two big things. One is it started from open sources, it started from a community. Second was trying to solve a real problem around as we move to micro-services’ architecture containers.

    How do you manage this stuff? How do you understand what’s happening? I remember Constance, you’re on a podcast we did together, we were talking about how do you know what happened where, what’s doing what, and what do these things do, to use generic terms, but what is it? So we’ve created this flexible, malleable component containerized architecture. 

    Now how do we understand it? We operate it and improve it over time. I guess the other thing we were talking to, you said software engineers, developers, operations people, they’re not into marketing hype. They’re pretty real folks. They want to know, are you solving a real problem for me or is it a checkbox and you’re calling it monitoring observability?

    So we have a pretty healthy skepticism in the market too that will fair it out, I think in most cases, what’s real, what’s not. Where the value is, to your point, Dotan. Okay, go on.

    Dotan Horovits: If I may just add on that, one of the benefits of the open source tools that we’ll talk about and standards, by the way, it’s not just tools, it’s also standards, such as OpenMetrics and others, is that it’s usually driven by real needs of real companies. Actually, today, I had an episode with a gentleman, senior SRE, site reliability engineer, from Google in charge of the entire identity services package of Google. 

    So when you log into your Google account, it goes through their micro-services. And many of the tooling that they use, and they’ve been using for a couple of decades, is the predecessor of what we currently talk about at the CNCF and many others. So obviously, Kubernetes itself, the containerization is a successor of Borg, their internal containerization orchestration system. 

    And then you have Google sensors that evolved into open sensors that now is emerged into OpenTelemetry together with OpenTracing. And you have the Dapper project that very, very fascinating research paper about distributed tracing that was, I guess, the forefather of Zipkin open source and the Jaeger open source and many others in the industry. 

    The nice thing is that Google’s scale is such that where we are at today was where they used to be a couple of decades ago. So in a way, these are well rooted within realistic problems at scale in micro-service architecture, in containerized environment, cloud native environments and technology. 

    So I feel much more confident basing Observability standards, tools and practices on the open source rather than on some vendor initiatives that again, might be inspired by some strategy and might have some forecasting behind it. But much less solid ground around Observability practices hands on.

    Constance C.: Two things I want to add to that, one is like, I think, I guess this is one of the things I want to keep on advocating for, is that I think we should get more just for nitpicking by cognitive doesn’t actually mean containerization. I think it’s just —

    Alan Shimel: And it doesn’t mean Kubernetes necessarily either.

    Constance C.: Exactly. Yeah, that was actually one of the points I was trying to make is cognitive is like any of the things there. But it’s more just I want us to dump that because a lot of people when they think about, oh, let me modernize my stack, I’m going to go straight to containerization. But that might not be the problem that they actually dissolve. 

    And so that’s just one thing I want everyone to be aware of when people are talking about cloud native is like, what is the problem I need to solve when you’re upgrading your stack, remodeling your house, thinking about that problem there? And I guess, I don’t know, Alan, if you want to talk to us now, but the vendor influence. The vendor influence is really interesting.

    Alan Shimel: I was going to bring that up. Let me jump into that. So let me just also lead off with this stuff. Look, Gartner has been doing a hype cycle for as long as I’ve been in technology, 25, 30 years. And the hype cycle is real. That is the way our industry works. Things get really, really overly high, they do the trial, disillusionment, and then they become productive, or whatever that end stage is. 

    What’s a little different about Observability is it’s in this new foundational era of open source, where you have, as I said, several really good open source projects that are driving so much of the Observability innovation out there. None of those projects are controlled by a single vendor. 

    They’re owned and managed by Linux Foundation and maybe some other foundations, which creates this whole golden era of coopetition, to your point Constance, among all the vendors. And this is a very interesting thing. So everybody starts with that same base. When Mitchell and I were doing vulnerability scanners 20 years ago, you could use the open source that was Nessus.

    Nessus was still open source then. You could use the Nessus scanning engine but there were proprietary scanning engines that people were using, some of them really based on Nessus some not. Not the case here. Here, you got a company like Logz, and Dotan, I don’t mean this in a bad way. But you got a company like Logz, for instance, almost their entire value prop is built upon open source. 

    Built upon that open source. I spoke to the Splunk people, their observability stuff, built upon open source. I’m just made mentioning that because we’ve got the two of you here, but there’s a lot of observability vendors out there. And they’re all built upon the same open source foundational tools. 

    I don’t know if we’ve had this kind of coopetition in a market like this in years past and in times past where, look, everybody starts at that same level and what you do with it. Who makes the best observability solution? I don’t know, I guess it depends on really what your needs are. 

    But it’ll be very interesting to see how in the real world laboratory of market conditions, people all come from the same open-source base, and what the value add is on top of that.

    Dotan Horovits: I would challenge two points there. First of all, we started the discussion from the CNCF and from the Linux Foundation, and we can add on top of that, the Apache foundation and others. But let’s bear in mind that many popular tools out there still are controlled by vendors. We can talk about the ELK Stack, ElasticSearch —.

    We can talk about the Grafana that is highly popular, and it’s controlled by Grafana Lab that actually rebranded itself to Grafana to be synonymous with the open source. So there is another beast out there, open sources that are very popular and very well adopted and very mature, but then again, not controlled by a foundation and this various implications. 

    I guess the most prominent implications are when these open sources are then being relicensed to some degree. So users one day wake up to find that the open source on which they built the entire system around is suddenly being pulled of the open source [crosstalk].

    Alan Shimel: One day we woke up and Nusses was an open source anymore, it wasn’t like that. But that’s exactly the point that I was making with foundational open source and why CNCF and these kinds of foundations —, Apache, whatever, it’s so great. And I’m not picking on Grafana, I happen to like Grafana, I like Loki as well, which is another Grafana project. 

    But the very fact is, if you have one company that’s controlling an open-source project, other companies are very often hesitant to contribute. They may use it for now, but they’re hesitant to contribute. And so in the long run, and I’ve been in open source, Mitchell will tell you, we’ve been in open source 25 plus years. 

    Over the long run the foundational model where you can have coopetition among various vendors prove superior to what I call the big brother open-source model where you have one vendor who owns that, and can pull it at any time or calls the shot. If I’m a VC-funded company, a unicorn or a public entity, and I’m investing 10s of millions, hundreds of millions of dollars, I want to be very careful that I am not dependent on an open-source project that is controlled by a single company. 

    This may be getting off observability and I’m sorry for getting on my soapbox, but I think this goes to the reason why CNCF and that community is so vibrant. It’s so much vitality, why the projects coming out of there are so popular and really changing our game. I look at observability because companies feel like everyone gets dealt the same hand, what you do with it is up to you. 

    No one’s going to pull the rug out from under you. And Constance, you’re on these boards so I’m sure you probably feel that way, but I’m interested in what you all think?

    Constance C.: The vendor has such a dirty word in like CNCF like KubeCon, Cloud Native Con, and open source. Okay, I’m putting on my vendor hat. Well, I’m actually putting on both hats, the vendor hat and also the contributor hat. One thing that the vendors have is that there’s a little bit of superpower. 

    And maybe we don’t as vendors or as leverages, is that we actually get to see a broad spectrum of how things are used. And if we’re doing a good job in terms of like, hey, we’ve seen these things like 10 times over and over so that we think this is really important and bring this to the community that could really help the community. 

    I think sometimes the messaging gets lost, because we’re just like, hey, we need this thing here and not explaining why about there. I think that’s sometimes why vendors get a bad name. It’s such a dirty word within open source, because you feel like, oh, you’re controlling how we want to use things, you’re going to make us buy these things there. 

    If you look at our stats, I can’t remember what the stats are on top of my head, a lot of our talks come from vendor companies because a lot of things we use, they’ve contributed so much time into it. I’m only saying this because I’m trying to make it less of a forbidden word. Because they do have valuable insights, and they do contribute a lot. 

    Then there’s also the end user part who has a lot of different use cases that vendors wouldn’t necessarily see. I also think for like some of the examples you mentioned, it might just be a matter of time before they move over to more CNCF. Because a lot of like, OpenTelemetry is still new, two-ish years. I guess, everything within CNCF are about six, seven years, so really, really new. 

    I think it’s just more like continuation of time, as CNCF is, as you can still see, it’s so new that it’s like iron itself out. It’s starting to be, you could say, the de facto place where a lot of people look to, to donate their projects and be a part of that community. 

    KubeCon and Con keeps getting crazy, crazy numbers of attendees and participation, it’s going to become more and more projects start coming to it, because there’s more of that trust, and it’s a little bit more of a matter of time. I like your, what was it?

    Alan Shimel: Coopetition.

    Constance C.: Coopetition. I’m starting to see it less and less. My scope is OpenTelemetry, I will say I wanted OpenTelemetry be like, we’re all vendors, you’re little skeptical. And as I’m leaving the GC now is like, oh my goodness, just seeing so many different people’s, so many different companies’ perspectives on what customers are targeting, what problem we’re trying to solve.

    And start to view these people at first, I’m like, who are you? It’ll now be like, setting up one on one fairly regularly. Talk about careers and life. We go to open source to collaborate with others who are working on projects that are similar to us or probably they’re similar to us. And this was like another avenue to see the more people and get to know them.

    Dotan Horovits: It’s important to say I work for a vendor — that also contribute to OpenTelemetry and to Jaeger and to others. So definitely I agree with you entirely that the know-how within these companies and the broad perspective of different use cases across different verticals, industries, geographies, gives a very valuable perspective.

    It just requires the right mindset of the vendor, because even if you look at the evolution, specifically around observability, players such as Datadog, New Relic and even Splunk started off with a proprietary agent that did very intelligent things. 

    I agree entirely that the ability with a proprietary agent to fetch the right telemetry, to enrich it with the right metadata to do the right aggregations, to fine tune it to the backend tool, storage and analytics and visualization, that’s perfect. But then again, it is very vendor locked. You can’t correlate across different vendors across different signals, assuming that not the same vendor serves all signals. 

    And that’s what the driver that brought us to OpenTelemetry. And gladly is to say, and by the way, today OpenTelemetry is the second most active project under the CNCF according to the DevStats, CNCF DevStat. It’s really amazing to see. That essentially brought the vendor to their realization that the locking through agents is no longer the future path. 

    And they’re aligning with OpenTelemetry and even contributing know how around the agents back to OpenTelemetry. And now the differentiating factor is within the collector, doing intelligent processing, aggregation and the power of visualization based on that. And that’s the right play, and that’s the play that evens the playing field because everyone gets the same raw telemetry and the raw signals. Now, it is up to you to draw the most value out of that and bring it to the end.

    Alan Shimel: That’s exactly what I was trying to say.

    Mitchell Ashley: With your point, yeah.

    Alan Shimel: That was exactly. I’ll take it one step further. It’s not just the CNCF. We do another video show on here with the open mainframe project of all thanks. Another Linux Foundation project. And it’s really an umbrella for 26 different open mainframe-related projects. It’s the same thing. You got Broadcom and IBM and Rocket and BMC, and many others all working together to be able to do like DevOps on mainframes, project Zoe and stuff.

    The CDF, Continuous Delivery Foundation, same thing. You got all the CD vendors in there.  Yes, there are the Grafanas there. Are the big brother open source players left there? This is the foundational era of all, and mean no offense, why like foundation. 

    But this is the foundational era of open source. You just said, Dotan, that’s it. That’s it. You can’t beat that. And that is why the Splunk and the New Relics and the rest of the world have come around to recognize that as well.

    Mitchell Ashley: Maybe [crosstalk] Alan. May be just kind of fanboy for a minute. What’s really awesome about this conversation is that we’re coming to it from multiple points. It isn’t just about let’s start an open source project, and that grows up into its evolution and becomes popular, etc. That’s one point of entry. There’s other points of entry, the proprietary. 

    We started here, now this has sprung up and we need to, maybe in some cases, companies are forced to. You have to, to be competitive, because something like OpenTelemetry has really become so foundational, to use that word, Alan, that that’s critical. You have to be part of that community, and maybe even have to contribute to be able to do that. I’m referring into a genuine vendor here. 

    And of course, tomorrow, someone else will start. And they’re going to start from where we are today. So what’s cool about this open source, it isn’t one thing, it isn’t one way of starting and one place of starting. You may be, like you were contributing to some early projects when you were at Lyft Constance. 

    Or you may be starting the next company, Dotan, that you’re going to build on OpenTelemetry and maybe one or some other projects, you’re going to join Jaeger, whatever, starts something new. But my point is, it’s not one thread through it, it is multiple threads. 

    But having an underlying activity that is bigger than any one of us, that we are contributing to, that it sort of advances the scrum as we move down the field, so we’re heading in a direction that sort of the rising tide lifts all boats kinds of things, yes, we’re all going to compete. And some people are going to only be interested in sort of their pure open source work. 

    And that’s what I’m going to do. And that’s how kind of a developer I’m going to be or what I’m going to contribute to. Others are going to say, how do I productize? How do I commercialize? How do I build something on top of that? Because I have some great ideas too, to develop it. And that’s what’s cool, how far open source is come from, when Alan and I tried to build some products on earlier open source stuff and to where we are today.

    Dotan Horovits: And that’s the beauty. The beauty is that, many talk about open source in the context of community in the greater good and everything, but then it’s important to say it is also a viable business model. It’s [crosstalk] one that you can build a viable business model, unlike some that claim that they cannot sustain it going forward. 

    And use that as an excuse maybe to take it off the open source or things like that. Actually, just look now, last week, or this week that chromosphere became a unicorn raising 200 million with a one point something billion dollar valuation. And that’s all based on M3DB or M3 that spun out of an internal Uber project, Uber engineering that developed its own time series database, scalable Prometheus, if you’d like, database in terms of time series storage.

    And now it’s a company in its own right with its own open source, in addition to a commercial offering. Now they’re expanding from metrics to also distributed tracing, as far as I understand. So it is happening. It is accelerating, actually, innovation, including business innovation, including new startups. It has sprung up around observability. 

    And it’s a very exciting thing to see that as the open source by polar signals that there is now around a continuous profiling in observability. You see that from all angles, and you see startup and innovation. And I think it’s fueled by and facilitated by the open source. And that’s something that they think is great.

    Alan Shimel: Absolutely. So here for me was the key sort of switch that switched, that enabled this. There was a time when we used to talk about the only open source business model that ever succeeded was Red Hat. That was a common thing you used to hear. And why? Because you look at some of the early open source commercial models, and they were about what we offer support and training. 

    And that was like, how do I monetize open source? You do support and trainings really. Well, that’s not a great model. That’s been proven, not a very lucrative model. Then we started doing some open core, premium modules. So I’m going to give you bare bones and if you want the good stuff, you pay me for it. And that was the model. 

    And look, there’s still companies doing that sort of model, it’s viable. Some people have problems with it, is it really open source? I think it is, but another story. But today, what we have is, Dotan, what you’re talking about. And I think, frankly, SAS has been the great enabler of this. Because now, I can take this open source project, set it up as a SAS app that’s going to do it for you. 

    You know what? People understand that these things cost money to develop, maintain, and deliver. And if I’m getting it as a SAS app like that, and I’m paying monthly or subscription, whatever your subscription is, that’s the way business is done. That’s the way software is done today, whether it’s observability, or anything else in this cloud native world. That has been the enabler.

    Yeah, it’s open source, but I’m getting it as a SAS app, this is what it cost, and I’m good with that. I don’t feel like I’m getting something that I should be getting for free. Ron Gula, who started Tenable Network skewed. I don’t know if you guys know Ron. But Ron always used to say, look, with open source that’s free as in freedom and free as in beer. 

    We’ve concentrated on the free isn’t freedom with observability. It’s not necessarily free as in beer. And I think that’s a key mind view that we need that’s enabled, quite frankly, Logz success. Dotan, right? People say yeah, maybe built on open source, but I’m getting real value out of it. And they’re packaging it all up for me and building on top of that. I’m willing to pay for that.

    Dotan Horovits: I would like to even reiterate what Constance said before because when you look at the vendors, the vendors actually bring value here. The vendors, say, okay, I will make the open source accessible to you. But then again, open source projects, and Constance probably can say a lot about that from Envoy and others, very focused on the core functionality, the core value proposition.

    But for a much more mature organization, it needs an entire wrapping around it that provides DSSO and the authentication, the certifications and all the administrative aspects. The open source developers are not even bothered with that. Even a more powerful UI and an accessibility and user management control agents who are back whatever. 

    And this is the wrapping that vendor with again, vendor that serves enterprises and knows all these needs can augment the open source with these capabilities. I think Splunk is doing it magnificently with enterprises and Logz.io and many others.

    But the idea is that okay, you have the open source that you know and love but the open source beyond having it in a managed version also needs some additional layers to get it to be a commercial grade or enterprise grade. So it can actually be acquired that whatever you run is certified, meets the criteria that you need to run it within your organization at scale. 

    Constance C.: So I guess I’m going to say I’m very disappointed all three of you. I guess you didn’t watch my keynote.

    Alan Shimel: No, I didn’t. I apologize. But tell us.

    Constance C.: Okay, so I don’t necessarily know if it should be offered. Some things shouldn’t be offered as like the vendor implemented for you. But the main thing, one of the points I was trying to make that CNCF is more than Kubernetes. That’s why I was calling out the Cloud Native. We don’t think the — hold it. But the next thing for CNCF you have to keep.

    Actually let’s use it like the hype train. CNCF is on this hype train. And it’s great, because it’s celebrating all this hard work. I’m going to say as a community, we actually deserve quite a bit of attention, because we’ve done a lot of great work. But for us to keep on having celebrations like KubeCon Cloud Native Con, and community based, and just having these fun discussions, we need to make sure that we think of CNCF as a whole product. 

    And each project needs to think as a product. And we actually need to start thinking about integrations. Because as CNCF as a whole, we have great projects here. But there’s no glue between them. And so once you drop one project, sometimes you get some other project for free. But if you’re trying to integrate multiple things at once, you get a hard time.

    You don’t get any freebies once you implement, say Kubernetes and you’re trying to implement Envoy, both things there. No overlap, and so we need to start thinking about from a more together. I don’t honestly know that should be offered to SAS, I actually think that – I know, there’s a bit of debate about should there be CNCF Stack. 

    But I think that we should probably actually have something easily set up 10 projects that were really great together. All look great. And that can help things there. The thing that I really like about where observability space maybe for the value add as vendors, is like the key distinction. For OpenTelemetry, it’s about instrumentation data collection. That’s where our boundaries are specifically set. 

    We will never do anything about visualization. We’re trying to make you gather your data and then after, you can choose how you want to interact with that. There’s a little bit of that boundary that I think are being set around projects of where things go, because you can either try and almost be just kind of gobbling everything up, everything around you. Or you set your boundaries and then you have to let something else track there.

    Alan Shimel: Sorry, I think that’s a CNCF decision. They have to make, like for instance, how many service match projects do you are under CNCF.

    Constance C.: Well, so that’s a no kingmakers rule. Like we do have an explicit rule about there being no kingmakers, because we do want to offer things right. Because I’m biased that I love Envoy, but for some people Linkerd works great. And that’s amazing because there’s all these things there. Some people really love Prometheus, I think open source is great. 

    Metrics is not necessarily equal there. There should be different option because things work better for other there. But it doesn’t mean that we can’t maybe say have like, I don’t know, a few CNCF Stacks. Like these 10 things work great here, these 10 things work there. And this is actually going to be more on the engineers, because CNCF does have more of the model of the open governance so you choose your own adventure. 

    But then you see a little more of, it’s this great thing about open source, you choose your own adventure but we also need to think a little bit more like PMS, because marketers and salespeople are thinking about what the future will be. Because if we just implement the next thing, like go faster build all those things there, you’re not going to have sustainability to project.

    Because I think the thing that we’re taught, everyone’s like, okay. This is almost sold as this is the last big project you’ll ever adopt. 

    Alan Shimel: Agree.

    Constance C.: But we’re not. And if we don’t think about the future, sorry, everyone, you’re going to have another rewrite in 5 to 10 years.

    Alan Shimel: Guys, we’re about out of time, I got to ring the bell here as I said in the beginning of our talk. But you know what, look. I didn’t take the prerogative of taking the last word because I’m the host. We are in a golden age of open source innovation right now. And we have been for the last couple years. And I think CNCF in similar kind of foundations have ushered this golden aged. 

    And there’s a lot of companies including Logz and Splunk, and many, many others who are constantly innovating and pushing what we can do off of this foundation, and I mean foundation of open source. And I don’t know how long it’ll last until we go into a post-cloud native world, I don’t know. I may not be here, but I do think when we look back on this, it is a golden age.

    And it’s going to be very interesting to see what the next short-term, because who knows long-term, what the next short-term future holds and especially for observability. Observability is kind of the poster child right now for what we can do. That’s my poster post right there. Anyway, on that note, I think we’re going to close this one out. 

    Constance, thank you so, so, so much for joining with us today, this was great. I apologize, I had that little network outage there and I don’t have you guys up on my usual monitor. So I keep looking over here to my computer. But thank you for joining us. Dotan, as always, a pleasure to have you on my friend. I look forward to being able to see you soon. Yep. And then of course, Mitchell. You want to take the last word, Mitchell, and we’ll wrap it up.

    Mitchell Ashley: I’m going to leave it right where you said it. You just said it perfectly. Let’s end there.

    Alan Shimel: Okay, then. Hey, we’ll be back in just a couple of weeks, two weeks I think, with our next Observability At Scale. It’s a great series if you’re interested in observability. Check it out. But for now, this is Alan Shimel, TechStrong Group. Have a great day.

    [End of Audio]