devopsy.kinsta.cloud

Tag: metrics

  • OpenTelemetry Project Maintainers Add Code Profiling Capabilities

    OpenTelemetry Project Maintainers Add Code Profiling Capabilities

    This week at the KubeCon + CloudNativeCon Europe conference, the maintainers of the open source OpenTelemetry observability agent software project being advanced under the auspices of the Cloud Native Computing Foundation (CNCF) revealed they have added profiling capabilities to enable DevOps teams to identify the root cause of issues down to a specific line of code.

    Austin Parker, director of open source for Honeycomb and a maintainer of OpenTelemetry, said while several providers of IT observability platforms have in the past year added code profiling capabilities, it makes more sense to provide this as a capability surfaced by OpenTelemetry in a way that any IT platform can easily consume.

    The overall goal is to reduce the amount of time it takes to troubleshoot IT issues by identifying not just the code at issue but also the development team responsible for it, noted Parker.

    OpenTelemetry defines a format that makes it simpler to centrally manage the collection metrics, logs and traces using open source software. In general, OpenTelemetry continues to gain traction as DevOps teams look to rationalize the amount of commercial agent software they need to deploy, update and manage.

    In effect, OpenTelemetry is aggregating agent software that, in terms of observability, provides no differentiated value, said Parker. That’s critical because as application environments become more complex, it’s all but impossible to manage them without being able to broadly apply the agent software required to instrument them.

    In the future, OpenTelemetry is expected to add additional capabilities such as data pre-processing, filtering or routing to redact sensitive data in a way that still fosters observability.

    Ultimately, as OpenTelemetry becomes more widely employed, it makes it significantly easier for developers to understand how their application is performing, noted Parker. Over time, armed with those insights, developers will build and deliver higher-quality applications, he added.

    It’s still early days as far as adoption of OpenTelemetry is concerned, but it’s clearly well on its way to becoming a de facto standard for enabling observability. The number of developers who have access to not just metrics and logs but also traces should steadily increase as the cost of instrumenting code declines.

    Of course, finding a way to collect that data inexpensively will also be crucial for applying artificial intelligence (AI) to DevOps workflows. After all, before an AI model can be trained, it needs access to verified data.

    In the meantime, DevOps teams might want to consider at what rate they can replace existing commercial agents deployed across the enterprise. Much of that agent software is more mature than OpenTelemetry, but over time, the economic arguments for using open source agent software become compelling. Many of the providers of observability platforms have already made support for OpenTelemerty available by default.

    Regardless of approach, the need for observability has always been apparent. The issue is that there is a world of difference between monitoring a pre-defined set of metrics and being able to launch queries to identify the root cause of an issue that metrics alone are not going to be able to surface.

  • Sumo Logic Eliminates Ingest Fees for Observability Log Data

    Sumo Logic Eliminates Ingest Fees for Observability Log Data

    Sumo Logic this week announced it will no longer charge a fee for ingesting log data into its observability platform to encourage DevOps teams to apply analytics more deeply.

    Michael Cucchi, vice president of product marketing for Sumo Logic, said the Sumo Logic Flex Licensing plan eliminates what amounts to a tax for using an observability platform. Flex Licensing is being made available immediately to new customers and will be offered to existing Sumo Logic customers later this year. There are no hidden monthly charges, feature restrictions, performance tradeoffs or user limitations being applied under the plan.

    The licensing plan only applies to log data, but DevOps teams will be able to ingest all structured, semi-structured and unstructured data for a lower cost that Sumo Logic is working to store more efficiently, said Cucchi. The overall goal is to make available a disruptive approach to licensing that lowers the total cost of observability, he added.

    Storing log data is going to be increasingly critical as organizations begin to apply multiple types of artificial intelligence (AI) models to further automate IT operations, he added. Those AI models promise to make observability platforms more accessible because algorithms will be able to surface issues that need to be addressed. Historically, the value of an observability platform has been tied to how proficient a DevOps team became when using its query language.

    IT teams that have adopted observability platforms have been, to varying degrees, limiting the amount of data they collect and retain to keep storage costs from exploding. The challenge is that as DevOps teams deploy more microservices-based applications, the amount of log data generated has exploded. Determining the root cause of any issue involving those applications can be problematic if log data isn’t readily available. Log data is, after all, the atomic unit for observability, noted Cucchi.

    In general, IT environments are becoming too complex for humans to manage without the aid of observability platforms augmented by AI models, noted Cucchi. Each application environment is fairly unique, so it is pivotal for those AI models to be exposed to as much data as possible to ensure optimal results.

    It’s not clear whether observability platforms might one day replace the need for many of the monitoring tools that IT teams rely on today to track a set of pre-defined metrics. However, one thing that is certain is that observability platforms, at the very least, present an opportunity to rationalize some of those tools. The biggest challenge is finding ways to fund the acquisition of an observability platform in the first place, which then creates an opportunity to consolidate those tools later.

    Given the complexity of highly distributed computing platforms, it’s only a matter of time before most IT teams have some type of observability capability in place. Of course, observability has always been a core DevOps tenet. The trouble is that the tools being used to achieve it are legacy monitoring platforms that don’t provide the level of depth required to successfully manage modern application environments.

    Image source: Joshua Sortino via Unsplash.

  • New Relic Streamlines Observability Data Management

    New Relic Streamlines Observability Data Management

    New Relic this week made available a live archives capability that provides instant access to historical logs for up to seven years and eliminates the need to rehydrate, reload, re-index or move data.

    At the same time, the company also announced its observability platform is now integrated with infrastructure and observability tools provided by Sentry Software to help IT teams reduce costs and collect additional sustainability metrics alongside logs, metrics and traces.

    Jemiah Sius, senior director of developer relations for New Relic, said overall, New Relic is moving to make it simple to analyze and store telemetry data regardless of when it was collected. For example, rather than requiring historical data to be accessed via a cold storage service in the cloud, the New Relic platform now makes that data available directly within the database that is at the core of its observability platform.

    That approach also provides the added benefit of not incurring ingress and egress fees for moving data in and out of cloud storage services, noted Sius.

    IT teams can simply create a New Relic Query Language (NRQL) to identify which logs to route to a live archive in as little as 30 seconds without having to add an additional log collector.

    The overall goal is to make it simpler to investigate incidents faster, given the urgency that is created whenever there is an outage or audit request that needs to be immediately addressed, noted Sius. That’s critical because DevOps teams are always for time, he added.

    In general, New Relic is reducing the overall toil associated that many DevOps teams encounter today when managing telemetry data. Rather than being confronted with a complex set of data engineering tasks, telemetry data from multiple applications and infrastructure platforms can automatically be collected, analyzed and stored.

    It’s still early days as far as the adoption of observability platforms is concerned, but there is already no shortage of options. The challenge is that in addition to logs, metrics and traces, IT teams are now starting to also track cost and energy consumption metrics. Observability platforms provide the means to centralize telemetry data that is only going to increase in volume as more applications are deployed. The challenge has been framing the queries needed to surface meaningful insights. Fortunately, with the rise of generative artificial intelligence (AI) tools that make it simpler to create queries, the overall level of skill required to invoke an observability platform is dropping. In effect, generative AI is making it possible to democratize DevOps best practices.

    Ultimately, existing DevOps teams should be able to manage highly distributed application environments without having to hire and retain a small army of additional software engineers. The issue is that, in many cases, the size of the overall code base is increasing rapidly as developers take advantage of various generative AI tools to help them write code faster. Hopefully, advances in AI will arrive soon enough to enable DevOps teams to cope with all that code before DevOps teams are inevitably overwhelmed.

  • Observe Inc. Adds Tool for Exploring Distributed Traces

    Observe Inc. Adds Tool for Exploring Distributed Traces

    Observe Inc. this week added a Trace Explorer tool to its namesake observability platform that makes it simpler to search, analyze and visualize billions of traces.

    Traces are being used with greater frequency to track how resources are being invoked across increasingly distributed computing environments. That capability provides DevOps teams with a level of insight into workflows that goes well beyond the logs and metrics that almost every IT team routinely collects.

    Observe Inc. CEO Jeremy Burton said the challenge now is providing IT teams with tools that make it simpler to affordably manage traces at scale to reduce mean time to remediation (MTTR). The Observe platform enables IT teams to store the traces they run in a data lake from Snowflake for 13 months versus other platforms that only store a trace for a few days, which then requires IT teams to rerun the same trace multiple times each time they launch an investigation.

    The overall goal is to encourage DevOps teams to launch more investigations to improve application performance by making traces readily available using low cost S3 storage from Amazon Web Services (AWS) that is accessed via the Snowflake data lake that makes it simpler to apply artificial intelligence (AI) to telemetry data.

    That approach makes it economically feasible for DevOps teams to not just resolve an incident but also provide the level of investigation required to accurately identify its root cause, said Burton. Because the cost of storing telemetry data is so high, DevOps teams that employ other observability platforms are not able to thoroughly investigate incidents because they lack access to telemetry data that their organization decided not to store because of cost concerns, he added.

    Most organizations are still mastering the nuances of observability platforms that enable them to query telemetry data that, in addition to being easier to collect using open source OpenTelemetry agent software, is also increasing in volume thanks to the rise of microservices-based applications. Each microservice now generates its own telemetry data that needs to be analyzed and stored. As microservices make application environments more complex, the need for tools to analyze all that telemetry data becomes more pronounced.

    As businesses become more dependent on software to operate, it’s now more than apparent that legacy approaches to monitoring IT metrics are no longer sufficient, said Burton.

    It’s not clear how quickly organizations are adopting observability platforms, but there is already no shortage of options. Given the current state of funding for startup companies in the IT industry, IT teams would be well-advised to research the level of funding the providers of these platforms have before deciding which vendor to rely on, noted Burton. Last year, Observe Inc. raised an additional $50 million in funding.

    Regardless of the observability platform selected, the one thing that is certain is that observability platforms infused with AI are changing the way IT is managed. The issue IT teams often encounter now is convincing the organization to fund the acquisition of these platforms at a time when budget resources in many organizations are already being stretched thin.