devopsy.kinsta.cloud

Tag: logs

  • Why Observability is Important for Development Teams

    Why Observability is Important for Development Teams

    Observability, which allows development teams to understand better how their systems behave in production, is critical to successful software delivery. According to Gartner, by 2026, 70% of organizations that successfully applied this practice will achieve shorter latency for decision-making, enabling competitive advantage for target business or IT processes.

    Observability allows development teams to identify and diagnose issues quickly to make decisions that improve overall software delivery performance. Let’s take a closer look at why this practice is critical for development teams.

    Differentiating Observability and Visibility

    To better understand observability, it’s essential to clear up a common misconception – that observability and visibility are the same. Two critical differences exist between the two. Visibility allows development teams to look at the surface-level aspects of an application or system. In contrast, observability provides deeper insight into what an application is doing in production. Being equipped with a deep knowledge of observability allows developers to be more productive within their respective organizations.

    Impact on Developer Productivity

    With observability, developers can identify and diagnose issues within applications quickly, giving them more time to fix bugs and time for resolution, which are both critical success factors for software delivery. Observability allows organizations to quickly identify and respond to potential problems, such as operational issues or security breaches, before they become issues that impact customers or users. While there is no such thing as perfect software, observability helps developers quickly find and fix issues, bottlenecks, inefficiencies, etc., so they can get back to doing what they love: Creating new software.

    With the benefits of observability, developers naturally gravitate toward tooling and technology. There are many tools available for developers to take advantage of, and development teams must learn how to use these tools to integrate them into their applications and services. Let’s explore some of the more popular options.

    The Shift to Cloud-Native Tools

    The shift to cloud-native tools is driven by the increasing popularity of cloud computing and its benefits, such as scalability, cost-effectiveness and flexibility. The move to cloud-native has made an enormous impact on observability efforts. Having good tooling and processes available positively impacts observability efforts because it’s a critical aspect of the developer communities’ systems. Overall, the shift to cloud-native tools is driven by the need for organizations to take advantage of the benefits of cloud computing and the need for more efficient and flexible deployment and scaling of applications.

    Some of the most common tools in observability include:

    These tools offer developers greater insight and knowledge into how applications behave. The cloud-native approach necessitates a new way to aggregate all of these operations.

    Conclusion

    With observability, developers can understand how their systems work in production. These systems act in different ways depending on the environments they run in. Therefore, organizations must do more to provide developers with sufficient observability because of the limited visibility into the overall system performance.

    Observability allows developers to act quickly and remedy issues before they go into production and organizations can better measure success relative to the application’s performance. Hours of developers’ time are spent manually debugging and troubleshooting when organizations could have a system in place to do this automatically and allow developers to get back to what they love doing most: Creating software. Observability helps organizations make better-informed decisions, improve operations and performance and achieve their objectives more effectively.

  • Grafana Labs Acquires Pyroscope to Add Code Profiling Capability

    Grafana Labs Acquires Pyroscope to Add Code Profiling Capability

    Grafana Labs this week announced it acquired Pyroscope and plans to merge Pyroscope’s open source continuous profiling project with the open source continuous profiling database that Grafana Labs launched last year.

    Richard Hartmann, director of community at Grafana Labs, said after logs, metrics and traces, continuous profiling is now the fourth pillar of observability that enables DevOps teams to identify how much infrastructure resources any given piece of code is consuming. That capability is critical for everything from controlling costs to improving performance to limiting the amount of carbon generated by an application both before and after an application is deployed, he added.

    The resulting Grafana Pyroscope project will be natively integrated with Grafana Labs’ open source dashboard project to make it simpler to visualize profiling data and correlate it with metrics, logs, and traces. Grafana Labs is also committing to adding profiling capabilities to the fully managed Grafana Cloud observability platform.

    Pyroscope currently makes a plug-in for Grafana available in addition to a cloud instance of Pyroscope. Integrating Pyroscope with the Grafana Labs portfolio will make it simpler for DevOps teams to visualize issues before there is a major disruption to an application environment, said Hartmann.

    It’s still early days as far as the adoption of observability platforms is concerned, but it’s apparent that as application environments become more complex, the plethora of monitoring tools that DevOps teams rely on will need to be streamlined. Observability platforms promise to unify logs, metrics and traces in a way that makes it simpler to launch queries to identify the root cause of an issue. The rate at which DevOps teams will embrace observability will naturally vary, but the biggest obstacle might not be the platforms themselves. Instead, the issue is understanding what queries can help DevOps teams better understand the root cause of an IT issue. Visualization tools, in combination with code profiling tools such as Pyroscope, should make it simpler to surface those issues.

    Grafana Labs is also enabling organizations to better address sustainability concerns that have emerged as more policies are enacted to encourage organizations to reduce the overall size of their carbon footprints. The European Union, especially, has been enacting policies that provide organizations with a range of incentives to encourage greater efficiency from IT environments. It won’t be long before more DevOps tools and platforms provide greater visibility and control over the amount of infrastructure application code is consuming to achieve sustainability goals.

    In the meantime, overall observability should continue to improve as organizations move beyond continuous monitoring to gain insights into how application environments are behaving. As IT environments become more complex, tools that monitor a set of pre-defined metrics are no longer enough to ensure application resiliency. The challenge—and the opportunity—is to determine how to take observability to the next level before the next inexplicable application performance degradation.

  • A History of Distributed Tracing

    A History of Distributed Tracing

    Organizations are increasingly using distributed tracing to monitor their complex, microservice-based architectures. Distributed tracing has become essential in microservice applications, cloud-native and distributed systems. Microservices and serverless applications can grow exponentially, which makes observing them at scale very challenging. The traditional logging method becomes expensive, and if there’s an issue, time-series data can reveal symptoms but not the root cause(s). With modern usage patterns of Kubernetes service mesh like Istio and Envoy, for example, there can be billions of microservices calls daily.

    What is Distributed Tracing?

    Traces describe the progress of a transaction or workflow through a system. Distributed tracing provides visibility into service dependencies and their interdependencies by providing end-to-end visibility. Visualizing transactions in their entirety allows you to compare non-normal traces with normal ones to determine differences in behavior, structure and timing. Through distributed tracing, you can identify bottlenecks in your system. As a traced request flows through a network, it can be tracked. 

    As requests flow from frontend to backend and to databases, traces provide diagnostic techniques that reveal how a set of services coordinate to handle an application request. Using distributed tracing, developers can troubleshoot requests with errors and high latency. A single trace typically shows the activity of a request or transaction within an application. A trace shows what has changed at each step. As you aggregate the traces, you can see which backend service or database has the most significant impact on performance. 

    The History of Distributed Tracing

    Dapper, a large-scale distributed systems tracing infrastructure, was introduced by Google in 2010. Two years after Dapper was made public, Twitter open sourced Zipkin, which was designed for application performance tuning. Zipkin was the first open source distributed tracing project. Zipkin trace data can be collected and visualized using a UI. In 2015, Uber launched Jaeger, which was inspired by Dapper and named after the German word for hunter. Later in 2017, Uber published a blog post, called Evolving Distributed Tracing at Uber Engineering, explaining the reasons for the architectural choices in Jaeger. In addition to creating Jaeger, its author, Yuri Shkuro, wrote a book about distributed tracing called Mastering Distributed Tracing. 

    In 2016 Ben Sigelman, founder of Lightstep, wrote a blog post called Toward Turnkey Distributed Tracing, describing OpenTracing as a single standard. Some people refer to this as the OpenTracing Manifesto. OpenTracing allows developers of application code, open source packages and open source services to instrument their code without binding themselves to any particular tracing solution. The goal of OpenTracing was to solve a standardization problem. Trace context must pass through all the components, including application code, dependent libraries, standalone open source services (Nginx, MySQL) and other vendor-specific libraries and services, to collect a complete distributed trace. Collecting a full path without a standard API to define the collection and passing of trace context is difficult. OpenTracing aims to solve this problem by defining a standard API that can be implemented by components from different tracing solutions, allowing the collection of end-to-end tracing data. The Cloud Native Computing Foundation (CNCF) accepted OpenTracing as its third project in October 2016. Two months later, OpenTracing 1.0 was released. After OpenTracing, Jaeger joined the CNCF. 

    W3C tracing context specification was proposed in November 2019, bringing distributed tracing closer to standardization. Over ten years, distributed tracing evolved from one paper to an active community. With standardization on all the layers, it is moving from just tracing to overall observability, ranging from latency optimization to root cause analysis and application performance management. It is moving from a single backend system to an end-to-end solution that spans multiple systems. 

    OpenTracing and OpenTelemetry merged in 2019. Using OpenTelemetry, distributed tracing can be implemented end-to-end. It released version 1.0 in 2021. CNCF’s OpenTelemetry project is one of the fast-growing projects, and OpenTelemetry has become the de facto standard for tracing, metrics and logging.

    How it Works

    Traces in distributed tracing consist of tagged time intervals called spans. It is possible to think of a span as a unit of work. Traces are directed acyclic graphs (DAGs) composed of segments known as spans. According to OpenTelemetry, There are four major components of a trace:

    • traceID
        • A unique 16-byte array to identify the trace that a span is associated with
    • spanID
        • Hex-encoded 8-byte array to identify the current span
    • Trace Flags
        • Provides more details about the trace, such as if it is sampled
    • Trace State
        • Provides more vendor-specific information for tracing across multiple distributed systems. Please refer to W3C Trace Context for further explanation.

    The spans in a trace consist of contiguous segments of work that are named and timed. There are start and end times for spans, as well as metadata such as tags or logs that help classify them. Parent-child relationships may be present between spans to show how particular transactions traverse the application’s numerous services and components. 

     

     

    A trace represents an end-to-end request; it can contain one or more spans. Spans indicate work performed by a single service with associated time intervals and metadata. The purpose of traces is to provide a request-centric view of spans through tags. Despite microservices enabling teams to work independently, distributed tracing offers a central resource that allows all teams to understand issues from a user’s perspective.

    Conclusion

    In distributed systems, response latency can have a significant commercial impact. Identifying bottlenecks and understanding how a request moves through a complex system is complicated. Other techniques, like logging and monitoring metrics, don’t provide insight into distributed applications such as those created with microservices architecture. Several open standards and tools are emerging within the distributed tracing space, like OpenTracing and commercial tools that could compete with existing APM solutions. Implementing distributed tracing for modern cloud-native services poses several challenges. OpenTelemetry tracing is necessary for processing high volumes of trace data while generating meaningful insight.

  • New Relic Reports Major Spike in Volume of Log Data

    New Relic Reports Major Spike in Volume of Log Data

    New Relic today shared a report based on anonymized data it collects that showed a 35% increase in the volume of logging data collected by its observability platform.

    The report also identified logs generated by NGINX proxy software (38%) as being the most common type of log, followed by Syslog (25%) and Amazon Load Balancer (20%).

    In addition, Fluent Bit is the most commonly used open source processor and forwarder tool (38%) among IT teams that have adopted the New Relic platform. Another 16% are using New Relic infrastructure agent, followed by 14% that send log data directly to New Relic via an HTTP endpoint.

    The report also found 50% of all logs ingested by language agents come from Java applications, followed by .Net (26%), Ruby (22%) and Node.js (2%).

    Jemiah Sius, director of developer relations at New Relic, said as more cloud-native applications are deployed, the volume of log data that is collected is only going to increase as open source OpenTelemetry tools are more widely adopted. In addition, IT teams will also be collecting metrics, traces and events to better pinpoint the root cause of issues using observability platforms such as New Relic One, he added.

    Amazon Web Services (AWS) is now, of course, one of the biggest sources of log data. The New Relic report noted that the serverless AWS Lambda service is the most widely employed source of log data among AWS customers that employ the New Relic One platform. However, use of the AWS Firehose service for extracting, transforming and loading (ETL) data has grown sharply in the last year, noted Sius.

    The New Relic report found while only 32% of New Relic accounts that use AWS have adopted Firehose, there has been a 62% increase in adoption year-over-year.

    Log data is usually the first thing any IT team consults whenever there is an issue, but as the volume of log data increases it’s becoming more difficult to associate log data with specific IT events. The issue that each IT team will have to come to terms with is how much log data to store alongside metrics, traces and other data formats to help them proactively identify the root cause of an IT issue.

    Observability platforms, in the meantime, promise to make it possible to query that log data to identify relationships between events. The goal is to enable IT teams to investigate issues before they can disrupt IT services rather than merely reacting to events after they have unfolded, said Sius.

    The hope, of course, is that machine learning algorithms will soon automatically identify issues before most IT teams even know there is a problem. Given the overall complexity of modern IT environments, it’s unlikely most IT teams even know what queries to launch to identify the root cause of an issue. Instead, as machine learning algorithms become more familiar with what constitutes normal IT events, it will become simpler for those algorithms to surface the issues that are actually worth IT’s attention.

  • Splunk Adds Data Management Tools to Observability Platform

    Splunk Adds Data Management Tools to Observability Platform

    At its .conf22 event, Splunk today announced it is making it easier to both onboard data and then manage it across hybrid IT environments via the Splunk Cloud Platform.

    In addition, Splunk Enterprise is being extended to add support for Microsoft Azure with SmartStore for Azure to store cold data alongside existing support for Amazon Web Services (AWS) and Google Cloud Platform (GCP). There is also a developer edition of Splunk Cloud that enables developers to create and test their own applications, which is currently in preview.

    Garth Fort, senior vice president and chief product officer for Splunk, said the overall goal is to make it easier to automatically tune the parameters used to store data in the Splunk observability platform. As more organizations embrace observability as a DevOps best practice, many of them are encountering challenges managing the large volumes of data they are now collecting in the form of logs, metrics and traces.

    The Data Manager for Splunk Cloud Platform will initially be made available on AWS and Azure, with support for GCP to come later this summer. In effect, it provides IT teams with a control plane for data management that spans multiple clouds.

    Ingest Actions, meanwhile, makes it possible to apply granular controls that enable IT teams to filter, mask and route data in motion within the Splunk Platform or to external AWS S3 cloud storage services.

    Other capabilities Splunk unveiled at the conference include an expansion of Federated Search tools to enable a single pane of view across their entire Splunk ecosystem and a fully managed Splunk Assist service for Splunk Enterprise 9.0 that provides customers with deep insights into their security environment.

    Finally, the company has enhanced Splunkbase, a framework for accessing curated collections of more than 2,500 applications and integrations provided by Splunk and its third-party partners.

    As the pioneer of IT operations analytics, Splunk has a large footprint within enterprise IT environments. It is now looking to expand that base to become an observability platform provider. It’s still early days as far as the transition to observability is concerned, but the goal is to make it simpler for IT teams to query platforms and surface issues that could lead to an IT disruption before it happens.

    It’s not clear how many organizations have the skills and expertise required to formulate those queries, but over time it should become more feasible to augment an IT staff with machine learning algorithms capable of discovering those issues.

    In the meantime, IT organizations are collecting more data than ever about their IT environments. In theory, that data will provide the context needed to optimize application environments that get more complex with each passing day. The challenge, of course, then becomes finding a way to efficiently store and manage all that data both before and after it has been analyzed.

    Regardless of the approach to observability, the need to go beyond simple monitoring of IT environments has never been more apparent.

  • Opportunities and Challenges of Observability

    Opportunities and Challenges of Observability

    Observability platforms that unify metrics, logs and traces across applications and IT infrastructure are transforming how IT is managed. Most DevOps teams today are able to continuously monitor IT environments using tools that track a set of pre-defined metrics; observability platforms make it possible to aggregate data so that DevOps teams can launch queries that help them uncover the root cause of IT issues. DevOps teams leverage observability to allow them to understand what is happening within their applications more effectively and efficiently.

    Historically, applications were black boxes—meaning it was (and still is, in many cases) extremely difficult to know what is happening under the hood of the application.

    With the advent and subsequent growth of observability tools and techniques in the DevOps space, teams went from struggling to collect and understand the available data (which could vary from robust to almost non-existent) to easily aggregating data from numerous sources and being able to understand what that data means to being completely deluged with data—so much data it’s hard to keep up.

    Gather Data is No Longer Enough

    Josh Chessman, vice president of strategy and innovation at IT infrastructure monitoring specialist Netreo, explained that as applications become more and more complex, (e.g., running across multiple disparate environments with different technologies and in different locations) being able to collect data from these different environments is no longer enough.

    “Instead, DevOps engineers need to be able to collect the data and efficiently analyze it, too, to understand what is happening within the application,” he said. “Observability is the concept that drives this.”

    Chessman pointed out there are numerous observability issues DevOps teams face, but that they vary from organization to organization (and sometimes from team to team within an organization). 

    “Part of the problem is that many organizations simply lack tools to provide the data collection and analysis needed to move to an observability paradigm,” he said.

    This could be for numerous reasons, including budget (if you already have an existing tool, can you afford to acquire a new tool?), functionality (not all tools that claim to provide observability really do), implementation (this stuff is not easy and a poorly implemented tool could actually make things worse) and more.

    “One of the larger challenges I’ve seen is that organizations will go out and acquire an ‘observability’ tool and assume that is all they have to do,” Chessman said. “Unfortunately, that is not the case. Acquiring the tool gets you the technology, but the team must also be structured and ready to take advantage of the concepts and functionality provided by the tool.”

    GitLab’s senior director of product management Kenny Johnston explained that, on average, DevOps teams use five different observability tools which tend to inadequately connect developers to observability.

    In turn, developers struggle to respond to incidents swiftly while jumping across multiple tools and sources of truth and struggle to assign service level objectives (SLOs) and error budget definitions to the responsible product development teams.

    “To remedy this, we recommend adopting a unified software development solution that replaces the DIY ‘duct tape’ of products constraining developers and helps them deliver software more securely, at speed and scale,” he said. 

    Johnston said the most flexible and future-proof solutions today are open source tools like Prometheus and its ecosystem, as well as the OpenTelemetry toolset for more advanced use cases such as tracing.

    Prometheus has become very common in the developer space and, unlike other open source solutions or libraries, its format automatically emits metrics.

    “This allows for simple and future-proof interoperability, as one of the hardest things to change later down the line is code running,” he said. “Using open standards reduces the need for developers to rewrite and reintegrate the same type of code over and over again.”

    Observability Requires Understanding Your Goals

    From Chessman’s perspective, understanding what your goals are is critical for observability. 

    “Observability tools can do many things, but their primary function is to collect and analyze data,” he said. “While that is all well and good, if you do not define your KPIs from the beginning, all the collected and analyzed data is meaningless since you have no concept of what good and bad are.”

    Trying to define good and bad from the data collected is likely to result in failure because you are looking at where you are to figure out where you want to go.

    KPIs vary from application to application and, if you do not have them identified and well-defined before beginning down the observability path, it is that much harder, he said.

    Most tools (observability or otherwise) are capable of collecting hundreds if not thousands of metrics, but not all—not even most, in many circumstances—will be relevant to an organization. 

    “Instead of having to sift through the cruft to find the valuable data, it is better to start with identifying the valuable data and then figure out how to get there,” Chessman said. “If you don’t identify your KPIs in advance you could end up using the wrong KPIs for the wrong scenarios.”

    Johnston added that KPIs, like DevOps, are an iterative process and starting with KPIs helps developers create systems that can meet performance and availability expectations.

    “It’s important that DevOps teams continuously review their KPIs to allow for regular feedback on successes and failures,” he said.

    From his perspective, the best place to start for maintaining and improving observability is by storing observability definitions, instrumentation, SLOs, thresholds, incident response templates and dashboards as code.

    “This allows for rapid improvement via code changes by any team member, whether that’s a site reliability engineer (SRE), developer or platform team,” he said. 

     

  • Overcoming IT Complexity With Full-Stack Observability

    Overcoming IT Complexity With Full-Stack Observability

    Organizations embarking on digital transformation journeys tend to develop complex infrastructures. They update their legacy applications while adding multi-cloud, virtual and cloud-native capabilities. Eventually, IT pros find themselves managing diverse, complex and distributed networks including cloud, system, application and database infrastructures.

    To rein in the resulting complexity, organizations tend to amass monitoring and managing tools. The goal is to simplify systems oversight—but instead, silos tend to develop because teams use a wide variety of tools to manage their networks or infrastructures. 

    This piecemeal approach worsens operational blind spots and delays problem resolution. It also increases security exposures. Soon, overwhelmed IT pros can’t keep pace with app modernization or infrastructure dynamics because they’re awash in complexity.

    Though this scenario is common, it’s not inevitable. IT teams can ease their digital transformation journey by implementing an integrated and cost-effective full-stack, end-to-end monitoring service capable of overcoming complexity and silos. 

    Organizations require full-stack observability.

    How Observability Differs From Traditional Monitoring

    Observability goes beyond traditional monitoring. Traditional monitoring helps IT organizations understand the actual state of their infrastructure and applications. It captures and processes volumes of infrastructure and application telemetry data and notifications and displays which components are up, which ones are down and which ones have changed. 

    Usually, traditional monitoring focuses on a specific network, cloud or infrastructure. It tracks application and infrastructure elements so IT pros can identify anomalies and investigate problems as they arise.

    Monitoring relies on metrics-oriented dashboards built to assess telemetry data against manual or basic statistically relevant thresholds. Monitoring tools are invaluable, but they don’t offer cross-domain correlation, service delivery insight, operational dependencies, or predictability. Modern systems have complex multi-cloud environments and a deluge of telemetry data: This counts as a shortfall.

    Observability does more. It measures the internal states of systems by examining the outputs and looks at applications and systems in their entirety—from the end-user experience to server-side metrics and logs.

    But observability also includes monitoring as a critical element. To gain observability, you must first collect information through monitoring. Observability uses the insights and metrics generated through monitoring to understand what’s causing the issue at hand. 

    Monitoring aggregates and displays the data to determine whether the systems are operating as expected. The analysis of this information is compared with expected outcomes and objectives. This allows IT pros to understand the state of their infrastructure and applications. 

    Silos are thus avoided because complex environments are viewed in their entirety.

    Observability in Action

    Once in place, observability allows IT organizations to continuously improve performance, availability and digital experience across complex, diverse and distributed hybrid and cloud environments.

    With observability, organizations can quickly find and resolve anomalies. However, full-stack observability goes beyond monitoring and expedited problem resolution: It supplies insights, automated analytics and actionable intelligence through cross-domain data correlation, machine learning (ML) and artificial intelligence for IT operations (AIOps). It works across massive real-time and historical metrics, logs and trace data.

    Observability goes beyond the silos and the piecemeal approach associated with traditional monitoring. And when observability isn’t limited—when it includes ML and AIOps—it uses the large volume of gathered data and supplies insights, automated analytics and actionable intelligence to help IT staff expedite problem resolution. It also enables ITOps, DevOps and security organizations to achieve consistent, optimized and predictable business service delivery with continuously improved digital experience and IT productivity. 

    The result is customers and employees benefit from better-run systems. The technology provides organizations of all sizes and industries with comprehensive, integrated, and cost-effective functionality through cloud-connected on-premises or software as a service (SaaS) deployment flexibility.

    When embarking on digital transformation journeys, organizations don’t need further complexity—certainly not while they update legacy applications and add a plethora of modern services and capabilities to their stack. The key to reducing complexity is observability.

    Observability simplifies the transformation process. It reduces operational noise to benefit ITOps, DevOps and security teams. They can become more proactive in issue and anomaly detection to achieve optimum IT performance, compliance, and resilience. With full-stack observability, any organization—no matter its size or industry—can reduce IT complexity while readying itself for digital transformation.

  • Log Management Challenges Grow With Observability Adoption

    Log Management Challenges Grow With Observability Adoption

    It appears that, in many IT environments, observability is becoming too much of a good thing. A global survey of more than 315 IT executives, cloud application architects, DevOps professionals and site reliability engineers (SRE) conducted by Dimensional Research on behalf of log data management platform provider Era Software, found 96% are witnessing an explosion of log data.

    The survey also found 79% of respondents reported the overall cost of observability and log management will skyrocket in 2022 if current tools don’t evolve.

    More than three-quarters of respondents (78%) also noted attempts to manage volumes of log data have had mixed or unwanted results such as an inability to access data. Nearly two-thirds (65%) are also evaluating their observability options while the other 41% are considering it.

    Stela Udovicic, senior vice president of marketing for Era Software, said the survey makes it clear that as organizations rely more on logs to analyze IT events, the cost of storing all that log data is becoming a significant challenge. Overall, the survey finds usage of observability tools and platforms has jumped by 180% as IT teams struggle to manage increasingly complex IT environments.

    On the plus side, more business users are benefiting from log data. A full 83% of respondents report that business stakeholders outside of IT use insights from log data, with 96% reporting log data is being used to solve business problems.

    Observability has always been a core tenet of DevOps best practices, but achieving it has always been a challenge. Monitoring tools are designed to consume predefined metrics to identify when a specific platform or application is performing within expectations. The metrics tracked generally focus on, for example, resource utilization. However, whenever there is an issue, it can still take days, sometimes weeks, to discover the root cause of an issue through what amounts to a process of elimination.

    In contrast, observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without relying solely on a limited set of metrics that have been predefined to monitor a specific process or function. DevOps teams can employ queries to interrogate data in a way that makes it easier to discover the root cause of an issue. Observability platforms correlate events in a way that makes it easier for analytics tools to identify anomalous behavior indicative of an IT issue.

    The issue that arises is that IT teams need to find a way to store all the data generated by the logs they need to analyze. Each IT team can, of course, could reduce the amount of log data they retain. However, there is always a concern that in the event of an incident critical log data won’t be available.

    Udovicic said most IT organizations have historically viewed storing log data as a necessary evil. The primary issue most of them encounter is the simple fact that log data is not easy to work with, she noted.

    Regardless of how organizations view log data, the amount of it that needs to be managed will only increase as DevOps workflows become more mature. The issue IT teams need to come to terms with now is finding a way to minimize the cost of storing it all.

  • LogDNA Survey Surfaces Observability ROI Concerns

    LogDNA Survey Surfaces Observability ROI Concerns

    A Harris Poll of 200 senior engineering professionals suggests it may be a while before most DevOps teams are able to achieve true observability in their IT environments.

    The survey was conducted on behalf of LogDNA, a provider of an observability platform, and found nearly three-quarters of respondents (74%) are struggling to achieve their observability goals despite investing hundreds of thousands of dollars. In fact, well over a third (38%) said they are already investing $300,000 annually on observability tools.

    Despite those investments, less than half of respondents are very satisfied with their ability to use log data. Key areas of frustration include the challenges of collaboration with colleagues spanning multiple teams (67%), the fact that the tools are not easy to use (66%) and the routing of security events (58%).

    Nevertheless, the bulk of survey respondents said they are optimistic about observability. A full 85% of participants said they believe true observability is possible.

    LogDNA CEO Tucker Callaway said the core issue that many organizations fail to recognize is that observability is really a data literacy and management challenge. Organizations are collecting logs, metrics and traces from a wide range of tools and platforms without thinking through optimal data pipeline construction. The result is an aggregation of data that has not been normalized to simplify surfacing actionable intelligence, he said. In fact, many organizations quickly discover that all they’ve really accomplished is increase their data storage costs, said Callaway.

    In the absence of that data normalization, Callaway noted it becomes exceedingly difficult for cross-functional teams to query all the data being collected within a repository.

    At its core, observability promised to make it easier to identify IT issues before they cause a disruption. Observability in one form or another has always been a core tenet of any DevOps best practice. Initially, DevOps teams focused on continuous monitoring as the most effective way to proactively manage application environments. However, it can still take days, sometimes weeks, to discover the root cause of an issue. Observability platforms make it possible to correlate events in a way that makes it easier for analytics tools to identify anomalous behavior that might be indicative of the root cause of an IT issue.

    In contrast, traditional IT monitoring focuses on predefined metrics to identify when a specific platform or application is performing within expectations. Observability platforms combine metrics, logs and traces—a specialized form of logging—to make it easier for IT professionals to interrogate data being generated by a wide range of DevOps tools and platforms.

    Of course, the assumption is that DevOps teams actually know what queries to launch to determine what the root cause of an IT issue might be. In the longer term, machine learning algorithms should make it easier to identify anomalies within IT environments. In the meantime, however, achieving observability still requires a fair amount of DevOps expertise that is not always readily available.

    Eventually, observability will become much more automated than it generally is today. As that transition occurs, IT teams should find the ROI of observability is steadily improving.

  • LogicMonitor Introduces Unlimited Log Data Retention

    LogicMonitor Introduces Unlimited Log Data Retention

    LogicMonitor today announced it will provide an option that allows IT organizations to retain an unlimited amount of log data on its software-as-a-service (SaaS) platform.

    Tej Redkar, chief product officer at LogicMonitor, said one of the issues that is holding back advances in observability is the cost of storing log data. LogicMonitor has decided to remove that issue by allowing IT teams to store log data on its LM Logs service for as long as they require, Redkar said.

    That data will also always be readily available as hot storage with no need to wait for data that has been stored offline to be rehydrated, he noted. Instead, that data is readily accessible alongside the metrics and distributed tracing data that LogicMonitor also collects, said Redkar.

    In general, LogicMonitor is making a case for centralizing observability for the entire IT organization via a SaaS platform infused with machine learning algorithms that automatically surface anomalies in real-time based on millions of events captured via log data. The issue many organizations encounter today is silos of decentralized observability data that are more complex to maintain while providing less visibility at a higher total cost, said Redkar.

    The LogicMonitor platform, in contrast, is designed to be equally accessible to, for example, a network operations team as it is to a DevOps team, he noted.

    Not every organization needs to store log data forever, but Redkar said as IT management continues to evolve, more organizations are moving toward storing metrics, traces and logs for longer periods of time. The rise of open source agent software is also making it less costly for organizations to capture that data in the first place.

    It’s not clear to what degree other providers of observability platforms will provide unlimited retention of data, but as the cost of storing data in the cloud continues to drop it’s apparent that billing IT organizations for storage costs is becoming untenable. There is always going to be some cost associated with storage, but it’s not enough to charge extra beyond the core cost of the service. The only thing accomplished by billing separately for storage is discouraging IT teams from storing the data required to both attain observability and better train the machine learning algorithms that surface anomalies.

    Observability, in one form or another, has been a core tenet of DevOps best practices for years. Initially, DevOps teams focused on continuous monitoring as the most effective way to proactively manage application environments. Observability platforms infused with machine learning algorithms make it possible to correlate events so that analytics tools can more easily identify anomalous behavior in real-time. Armed with those insights, it becomes a lot simpler for IT teams to resolve issues faster.

    In fact, there may even come a day when the so-called “war room” meetings that are convened to identify the cause of an IT issue via a painstaking process of elimination are no longer required. In the meantime, however, the total number of IT incidents that actually lead to disruption should steadily decline, even as IT environments become more complex.