devopsy.kinsta.cloud

Tag: observability

  • Help Developers Thrive by Finding the Right Platform

    Help Developers Thrive by Finding the Right Platform

    If our developers have what they need to create the best possible applications, they—and our customers—benefit

    Goji Investments was founded on the principle of democratizing access to real estate, business lending, renewables and other alternative investments. The idea is to allow financial service organizations to provide traditionally unavailable investment opportunities to private investors on a global scale. A critical component of the Goji plan for success is the investment platform. Not only do we make the application available as a white-label solution, it can easily integrate into other platforms via APIs. We wanted to create a modular platform that we could update in pieces when needed and easily scale up as clients came online.

    High-Performance Development Teams

    A major reason for Goji Investments‘ success in these first few years is our high-performance development organization, which is instrumental in building and delivering the vision of the investment products needed to generate new business. As we’ve moved from simply building our applications into making them more scalable, we’ve focused on getting the right tools in place for Ops and Dev. Of course, bonus—it makes developers happier if they have cool tools to use.

    One of the things we try to do is keep our application/platform deployment process as agile as the applications themselves—constantly looking to improve and making updates whenever necessary, whether needed to deploy new technologies and applications or based on feedback from the team. The idea is to keep our DevOps pipeline operating smoothly and efficiently.

    One of our key operational strategies is that if we make our developers happy at work, they will deliver better quality applications, which results in happier end users. Of course, developers like to create to come up with innovative ways to deliver great features. One roadblock to coding innovation is performing repetitive low-level tasks, so we’ve actually gone out of our way to automate whatever we can and free up Dev resources for more value-specific application development.

    Automating Deployments is a Start, but Just a Start

    We automate the deployment process through GitHub. Each developer pushes their own services into GitHub themselves, where it goes through a quick review process. After review (if it passes), it’s merged into the master. Any master merge triggers build pipeline execution (which is using ThoughtWorks GoCD), and this automation includes several rounds of testing: unit test and integration test followed by an automated acceptance test. Developers have the option of executing additional tests themselves when desired.

    Any new master is promoted into pre-prod, which mirrors production. When changes are finally rolled into production, we use HashiCorp Nomad’s blue-green deployment capabilities—if a problem were to occur (which means it was missed in official testing), Nomad takes care of any rollback needs.

    The stack is fairly straightforward: The applications are written in Java and run on a DropWizard framework in containers. Nomad is our orchestration platform. One of our foundational principles when selecting application and infrastructure technologies is the ability to automate our deployment processes. We’ve done a fairly good job of automating anything we possibly could, but automating monitoring was more difficult. Our traditional APM tool was fine for alerting us to issues such as an EC2 crash or a resource availability issue for a reboot, but if we wanted to actually troubleshoot the issue we essentially just had to examine logs and hope we could see something out of the ordinary.

    As much as everyone is excited about writing data into logs, combing through those logs to troubleshoot application issues is a waste of time and valuable developer resources. After all, operating 20 different microservices that cover everything from taxes and payments to customer notification required optimization of each element of the DevOps pipelines. The last piece we added was performance monitoring.

    The Value of End-to-End Distributed Tracing

    There’s a great line from Jurassic Park: “T-Rex doesn’t want to be fed. T-Rex wants to hunt.” Well, developers don’t want to be analyzing log files; they want to code. In fact, they want to be creating the next great feature. To get our dev team out of the weeds, we began thinking about a different way to get—and analyze—application performance data.

    We wanted a system that added observability, monitored performance, managed service levels and included visibility from our end user requests all the way into our back-end systems. Oh yeah, we also wanted all of that to happen automatically so that our developers never had to worry about anything other than their own features. Installation, agent setup, discovery, mapping, alert configuration and reporting—as much as possible without requiring human setup.

    We knew tracing was critical—so much so that we avoided missing any problems by specifying a 100% solution, wanting a complete trace of every request to the application.

    Troubleshooting Application Performance Problems

    The real reason you buy application monitoring software is to solve problems, so optimizing our ability to find bottlenecks and eliminate them is a key component of our monitoring solution. First, like the other aspects of monitoring, we didn’t want to have to spend time configuring thresholds for warning and danger alerts. Our need was for automatic anomaly detection with event analysis. Every little bit helps keep our dev team focused on dev, not analyzing data—especially in logs.

    Of course, sometimes, you can’t help it; you have to look at log data, but we wanted to be smarter about that, too. We wanted our application monitoring tool to automatically integrate log analysis tools so that all data could be analyzed at the appropriate level whenever needed.

    Don’t Forget Testing

    Sometimes, testing gets lost in the value of a good monitoring solution. While we weren’t necessarily looking for a testing tool, we knew if a monitoring solution could also operate effectively in test (that’s not just the ability to run in test; rather, I mean the ability to add value in the testing process), well, so much the better.

    It turns out that our applications are quite complex, and that complexity can create lengthier test cycles—commonly in this scenario:

    We recently released a new feature for distributing dividends. As you can imagine, this is a fairly important feature, so we wanted to ensure that it operated as desired and expected. We were especially concerned with being able to scale the feature appropriately. We set up a stress scalability test that took 20+ hours to run (essentially processing hundreds of thousands of payments).

    What we didn’t count on was having to stop some way into the test for someone to either submit an optimization update or actually fix a bottleneck—getting 10-12 hours into the test, then having to halt it and restart it—over and over and over again. We started thinking we might never finish.

    This is one of the pain points we hoped to solve with an automated way of monitoring. By pointing the automatic APM solution at our test environment, not only did we not have to configure anything (or even set up a special start command) but we also got almost immediate feedback on any change (both positive and negative). It also pinpointed the exact location in the code where the problem occurred, so the developer could almost immediately update their code and initiate a restart of the test. Finally, we could pick any two runs and compare their execution and performance to identify the real problem.

    Two things happened:

    • We were able to iterate different runs ~5 times faster.
    • We got such great bottleneck and debugging data from our solution that we cut the runtime of the test from 20 hours to less than one hour.

    Give Developers the Tools to Shine

    While all employees like to succeed, developers especially love when they can rise above their challenges and deliver high-performance code. We like to give our dev team the best possible tools to help them achieve this excellence. Approaching the decision process for selecting a tool for them should be handled no differently than thinking about your end users:

    • Identify pain points.
    • Find a resolution.
    • Execute the solution.

    For our development team, that meant relieving them of low-thought repetitive tasks that we could automate and stop making them spend their time culling through performance logs.

    Making the Right Choice??

    The long and short of it is that we replaced our legacy APM tool with a solution that delivers the automation we crave and eliminates the need for log deep-dive analysis to solve any problem. It also supports log analysis with various integrations, which was important to us.

    We get a trace of every single call, which is important to the Ops and customer support teams—and they profile every production process automatically, which gives our developers more than enough information to take action.

    Ultimately, we’ve accelerated our application delivery and update processes, streamlined testing and monitoring. And our developers are super happy that we give them the tools they need and then challenge them to make the best possible applications using those tools.

  • Grafana Labs Looks to Simplify Observability

    Grafana Labs Looks to Simplify Observability

    Grafana Labs, at its online ObservabilityCON conference today, unveiled a distributed tracing platform dubbed Grafana Tempo that makes it possible to leverage existing object storage platforms and services to analyze traces.

    At the same time, Grafana Labs announced version 2.0 of Loki, which normalizes different structured, unstructured or JSON log formats in a way that allows DevOps teams to extract additional labels as well as enable additional filtering and grouping. IT teams are also not required to define labels in advance and store those labels in the database.

    A Loki 2.0 query can also be used to directly generate an alert statement using a new distributed rules evaluation engine. Previously, Loki would have to have been configured as a Prometheus data source, which then would allow the distributed instance of Prometheus, dubbed Grafana, to generate alerts.

    Tom Wilkie, vice president of product for Grafana Labs, says Loki 2.0 also serves to make the database employed to observe events in real-time smaller, resulting in faster queries. The result is an approach to analyzing log data that is significantly less expensive than legacy platforms, he said, noting DevOps teams should expect to see Grafana Labs also use Loki to address security use cases.

    Wilkie said both Loki and Grafana Tempo are intended to reduce the barriers to achieving observability in a way that doesn’t require nearly as much time, effort and skills on the part of a DevOps team. Grafana Tempo supports the same Tempo data discovery engine employed by Loki and the open source Prometheus monitoring platform and the Grafana platform built on top of Prometheus. That level of integration makes it easier for IT teams to narrow in on a specific trace using open source tracing protocols such as Jaeger, Zipkin and OpenTelemetry. Users of the platform can jump from log to trace and back again easily, he said.

    There is also no need to set up and maintain a separate index that IT teams must maintain, Wilkie added. In addition, Tempo is compatible with object storage systems commonly found on cloud services, so the total cost of achieving observability drops significantly compared to platforms that rely on a proprietary database, he said.

    Despite observability being a core tenet of DevOps, it remains an elusive goal for most IT organizations. There is no shortage of tools for monitoring individual applications and systems, but aggregating all the data the tools generate into actionable intelligence requires a significant amount of time and effort. Loki makes it much easier to analyze the relevant logs at the right time in a way the average IT administrator can navigate without having to become a superuser of more complex platform, said Wilkie.

    It’s too early to say how the observability wars will play out. The good news from an IT perspective is at least now there is no shortage of options.

  • Cracking Production Performance Issues With Open Source Observability

    Cracking Production Performance Issues With Open Source Observability

    I would like to share a recent case study on our organization, Logz.io, which battled a very serious performance issue, the solution for which turned out to be a small change in code but with a huge impact on all of our HTTP endpoints in our platform.

    Among other topics, I’ll cover:

    • How we investigated and found the root cause.
    • Logs, metrics and tracing all together to have better observability.
    • How Jaeger, as a tracing tool, ease to visualize our potential bottlenecks.
    • How to correlate logs and traces.

    Performance: Our Wake-Up Call

    Some may say we are experts in observability and mainly in logging; after all, that’s what we do for a living. Once we encounter any performance issue, it should be no-brainer for us to fix it in no time.

    That might be true in certain use cases, such as whenever there are few slow endpoints and identifying the root cause via logs is easy. But in practice, logs are not enough when the situation is more complicated. We’ll see this later on.

    In our case, performance issues were reported especially during peak usage times—2-7 p.m. local time, overlapping between EU and U.S. customers.

    We use our own product, just like any other customer, so we saw on our internal Slack channels that employees were experiencing increased latency over the same period of time.

    Better Observability When Using Logs, Metrics and Tracing

    To correlate users’ complaints and actual monitoring data, we usually check all three main pillars of observability: logs, metrics and trace data. Let’s have a closer look at how we used each one of them in this case:

    Logs

    Our log analytics product is built on top of open source ELK stack.

    We use a custom field processingTime (the amount of time it takes for a backend endpoint to be processed) in our access logs and then visualize it over time in Kibana. That better picture can identify suspicious endpoints that take more time than anticipated in peak times. We can see below an example of an endpoint that consistently takes more time (ms) during the 2-7 p.m. window.

    In addition to the processing time each suspicious request took, we saw growth in the number of requests to the microservice handling our user session tokens. Likewise, we noticed a very high load of requests during those same peak hours.

    Metrics

    Our metrics product is built on top of open source grafana. In our backend infrastructure dashboards, we found consistent spikes in transaction commit size written into our distributed database (Galera). Those spikes correlated with the same peak times above between Monday and Thursday. Apparently, this might look pretty valid because these are our customers’ rush hours, but we’ll see later why it made us suspicious.

    Additionally, we faced significant replication latency in our database cluster during the same period:

    Distributed Tracing

    Last but not least, our tracing product is built on top of open source Jaeger. We checked for patterns of high latency by filtering out requests that took less than 5 seconds. Again, slow requests popped up more during peak hours.

    Each type of representation of data has its own benefits, so with using Logz.io it was very easy to correlate all data and navigate between them to find the performance bottlenecks.

    One of the must-have features to correlate logs, metrics, and traces is using the trace-id common tag. Alternatively, in lieu of it, another common tag such as request-id matches these requests throughout all of our microservices together.

    Performance Investigation Deep Dive With Jaeger

    As we already discussed, our goal was to find what made our application slower than usual during peak hours. We needed to trace our application flow to find out where the performance bottlenecks were. More importantly, we can track how we got there, i.e., find the request flows that led us to the suspiciously slow span. As modern microservice-based applications boom, it gets more difficult to trace things manually, by adding dedicated logs of before and after any needed code block. Moreover, it makes the code cluttered and unclear with unwanted log lines.

    In our case, Jaeger made our life much easier. Especially for a new developer in the company who was not fully familiar with all of the involved components, it was easier to understand the business flow by identifying long-running spans before changing any code (which takes much more time).

    So, if we could point to the exact places in our code where performance bottlenecks, it would be super easy to fix it in no time. However, our case was a bit different. Virtually every endpoint in the back end was suffering at peak times while firing a search request:

    Peak hours trace

    Normally with slow search queries the suspicion falls on the database, in this case Elasticsearch. So, we correlated the above span from the POST /services/query request with our logs and realized that the actual Elasticsearch query only took 15ms! So, we were able to rule that out.

    It’s important to emphasize that in non-peak hours, these microservices take 10 milliseconds at most:

    Off-hours trace

    As we can see, that looks way different from the example we showed before! So, why does it behave so dramatically worse in rush hours?

    The Root of Performance Issues: We Found It!

    We encountered pretty much the same slow pattern in all related endpoints when performing search requests. As a result, we were pretty confident that the root cause for all of it had to be in one of our core cache components. In other words, too many requests were delivered to our internal microservice (which serves the user session token object from the database) instead of getting it directly from the cache.

    This hypothesis also fits the scenario above since more active users implies more SELECT and UPDATE queries for each user session token at precisely the same time we saw latency peaks in our graphs.

    From there, finding the best solution was pretty straightforward and the fix took us just two lines of code! It was a matter of changing the configuration of our session token cache eviction policy.

    Boommmm

    The most exciting part of these kinds of fixes is to see how all the graphs change immediately. The writes and reads traffic to and from the database, plus the high latency in many requests, dropped as soon as we pushed the fix live to production. The impact was absolutely clear in our application dashboards:

    We could see the total size of transaction commits dropped right after the deployment.

    Also, we saw the total number of requests to the problematic service cut off by more than 80%!

    And most importantly, as soon as we deployed the fix, the latency dropped across all the impacted endpoints:

     

    For the coup de grace, we were waiting to hear from our customers any positive feedback. And we didn’t have to wait long—we immediately received high marks.

    Summary

    We still have a long way to fully instrument our system, but even with what we did so far we see that the efforts have paid off. We got a really cool and meaningful view of our system, which makes us better understand our business flows and, more importantly in our case, to quickly identify any bottlenecks.

    We already learned from our ongoing tracing adoption journey a few tips worth sharing:

    • Correlation between spans and logs is a must-have. The best practice for doing it is with a unique identifier such as trace-id or in case it’s missing with request-id. It brings a lot of value when you better want to examine a full request end to end by one click, which moves you to the respected logs page and vice versa. With Logz.io, you will get the log-trace correlation out of the box, which simplified things for us.
    • If your application is multi-tenant and you want a better view of traces of a specific customer, I’d recommend adding custom tags like customerId or userId.
    • If you want more finely tuned traces, consider adding custom spans for important pieces of application logic. Don’t just wrap every endpoint, which is the easy thing to do. If we suspect, for example, a slow query in the database, we can easily add in code for a custom span that will measure just this section and then it’ll be visualized under the parent transaction span, so we can see how much time it takes comparing the other spans.

    Having Logz.io made our journey much smoother—having logs, metrics and traces in one consolidated platform without the need to install and maintain it ourselves and with built-in correlations.

  • Splunk Adds Plumbr and Rigor to Observability Portfolio

    Splunk Adds Plumbr and Rigor to Observability Portfolio

    Splunk this week at its online.conf20 conference revealed it has acquired Plumbr, a provider of an application performance management (APM) platform, and signed a definitive agreement to acquire Rigor, a provider of synthetic monitoring and application optimization tools.

    Both platforms are destined to be integrated with the Splunk Observability Suite, a suite of IT management tools the company launched this week as well.

    Splunk CTO Tim Tully said the goal is to provide a full-fidelity platform that leverages machine learning algorithms to observe IT environments in real-time.

    Plumbr, which includes real user monitoring capabilities as part of its platform, will extend real user monitoring (RUM) capabilities, currently in beta, that Splunk has developed on its own.

    Tully said Splunk is also committed to making its platforms available across hybrid cloud computing environments by leveraging cloud-native technologies such as Kubernetes, Docker containers, Envoy, Istio and Terraform.

    Splunk has embraced observability, a core tenet of best DevOps practices, as part of an effort to provide a range of digital experience management capabilities at a time when many organizations are accelerating their investments in digital business transformation initiatives. Splunk last year acquired SignalFX for more than $1 billion to gain access to another APM platform that would expand its reach beyond its core IT operations analytics platform.

    It’s not quite clear how many platforms and services will ultimately make up the Splunk Observability Suite. The company is clearly moving toward a model for consuming its services based on value rather than the amount of data collected. Determining that value will naturally be different for each customer, but in the wake of the economic downturn brought on by the COVID-19 pandemic, the pressure to reduce IT costs has intensified. In many cases, the cost of acquiring an observability platform can easily be justified by eliminating the licensing fees organizations pay to employ a dozen or more tools from different vendors that typically are not integrated.

    Competition across providers of observability tools has also intensified. In addition to every provider of an APM platform recasting itself as a provider of observability tools, there is a range of startups that address all or some aspect of observability. A wave of consolidation among providers of observability tools and platforms is all but inevitable.

    Much of the focus on observability is being driven by the rise of microservices-based applications that have levels of dependencies requiring a level of context to successfully manage what isn’t possible to achieve by relying on legacy tools that monitor applications and infrastructure in isolation. The lack of integration is also one of the primary reasons why IT teams waste so much time in “war rooms” trying to deduce the root cause of an issue, which, once discovered, only takes a few minutes to fix.

    Splunk, of course, enjoys a considerable advantage in the observability wars. Its core IT operation platform is widely employed by internal IT operations and security teams. However, much of the transition to observability is being driven by DevOps teams who take the lead when it comes to instrumenting applications. Many of those teams are far less committed to any specific platform no longer how established it may be.

  • Splunk Dives Deeper Into Observability

    Splunk Dives Deeper Into Observability

    Splunk, during its online .conf20 conference today, launched the Splunk Observability Suite, which combines tools for infrastructure monitoring, application performance monitoring, digital experience monitoring, log investigation and incident response in a single offering.

    Kia Behnia, vice president of ITOps for Splunk, said IT teams can correlate metrics, traces and log data in real-time to go beyond simply monitoring IT platforms to also include the end user experience. As part of the Splunk Observability Suite, Splunk is making available in beta Splunk Log Observer, a cloud-based service for visually inspecting logs and sending messages to DevOps platforms based on events being observed by Splunk platforms, and Splunk Real User Monitoring, which extends the reach of Splunk Application Performance Management(APM) platform to add a module that tracks specific end user application experiences.

    That critical because without end user monitoring capabilities an observability platform will lack crucial context, said Behnia, noting observability needs to start with IT infrastructure and then extend out to encompass the actual end user experience.

    Splunk this week also announced enhancements to Splunk Cloud and Splunk Enterprise, which are now based on a version 8.1 of the core Splunk platform that is now generally available. The company is also making available in beta a Splunk Machine Learning Environment (SMLE) to makes it easier to build and operationalize machine learning models and algorithms.

    The Splunk Data Stream Processor (DSP) platform is also being updated later this year to add support for cloud services such as Google Cloud Platform (GCP) and Azure Event Hub along with support for lookups and machine learning algorithms.

    Splunk is also making available a beta release of a Splunk Operator for Kubernetes to simplify deployments along with making Splunk Cloud available on GCP.

    The company is also updating Splunk Connected Experiences to include a Splunk Augmented Reality capability that adds a remote collaboration feature that allows users in two different places to interact and visually compare data.

    Splunk TV, a mobile application for analyzing IT operations trends, is now also available on Google Android as well as Apple. Users can now also centrally control multiple TVs without having to be connected to the same network.

    Other additions to the Splunk IT Operations portfolio include Splunk Service Intelligence for SAP, which applies artificial intelligence (AI) to these environments; an update to Splunk IT Service Intelligence (ITSI) that adds predictive analytics tools; a Splunk Infrastructure Monitoring Add-On to integrate Splunk Infrastructure Monitoring gained via the acquisition of SignalFX; and the beta of Splunk IT Essentials to provide a more curated implementation of Splunk Cloud.

    Finally, Splunk has updated Splunk Enterprise Security and launched both Splunk Mission Control and the Splunk Mission Control Plug-In Framework. The framework makes it possible to combine third-party security tools with updates to Splunk security information event management (SIEM) and Splunk Phantom security orchestration automation and response (SOAR) platforms. Initial partners include Accenture Security, C3M, Crowdstrike, Netskope, Recorded Future, RiskIQ and Tenable.

    Put it all together, and it’s clear Splunk is now pushing to extend observability in every IT direction possible.

  • Observe Unveils Observability Platform Built on Snowflake DBaaS

    Observe Unveils Observability Platform Built on Snowflake DBaaS

    After raising $35 million in seed funding, Observe Inc. has emerged from stealth with early access to a namesake observability service that leverages the database-as-a-service (DBaaS) platform created by Snowflake Inc.

    Company CEO Jeremy Burton said rather than build another data lake platform, it makes more sense to make an observability platform available using a software-as-a-service (SaaS) application model based on an existing cloud service.

    Burton said Snowflake, which is based on a relational database running on the Amazon Web Services (AWS) cloud, has already proven it can scale to ingest logs, traces and metrics. The Observe platform leverages that platform to correlate that data to monitor application and infrastructure environments in a way that provides more context than legacy tools that monitor software components and infrastructure in isolation.

    The context is achieved by creating a “resource” to represent, for example, a user, a session, a shopping cart, a pod, a container, a help-desk ticket or a build. Observe then enables IT teams to also track relationships between resources in a way that the average IT administrator can query without having to tag each resource, said Burton. IT teams can also track how the attributes of a resource change over time, which Burton noted can be employed to reconstitute the state of an entire IT system at any point in time.

    That approach can be applied just as easily to existing monolithic applications and emerging microservices applications because all the data required is ingested into the Snowflake service without requiring IT teams to first instrument and then update each application, added Burton.

    The pricing model for Observe will be based on the amount of data stored in the AWS S3 bucket and the number of queries launched. IT teams will purchase Observe credits that are billed only when the platform is actually used, noted Burton.

    While observability is a core tenant of best DevOps practices, Burton said there is a need to democratize observability in a way that allows any member of an IT team to troubleshoot any issue. There shouldn’t be a requirement to assemble every member of an IT team to diagnose an issue, said Burton, noting over time, that just aggravates developers who don’t really need to deal with, for example, an infrastructure issue.

    IT teams will be provided access to dashboards that enable novice or intermediate users to quickly ascertain the root cause of an issue, while more seasoned engineers with be provided access to worksheets based on a spreadsheet-like interface through which they can manipulate data, said Burton.

    Competition among providers of observability platforms is already fierce. However, as organizations look to rationalize existing legacy monitoring tools in favor of platforms that add more context, the overall size of the observability market will be measured in billions of dollars. Each IT teams will need to decide what type of platform makes the most sense for them based on costs and their preference for one capability a vendor might have that another doesn’t provide.

  • Sumo Logic Extends Observability Reach

    Sumo Logic Extends Observability Reach

    Sumo Logic, at its online Illuminate conference today, announced the availability of a bevy of extensions to its observability and security information event management (SIEM) cloud services that span everything from hybrid cloud computing to cloud-native platforms such as Kubernetes.

    The company also announced the general availability of a free Software Development Optimization (SDO) service, a business intelligence application that integrates and analyzes data collected from multiple DevOps tools to provide insights into software development pipelines in real-time. Based on metrics defined by DevOps Research and Assessment (DORA), a research arm of Google, SDO comes integrated with Jira, GitHub, Jenkins, Bitbucket, PagerDuty and OpsGenie and can be extended to other DevOps tools.

    Sumo Logic also updated its Application Observability service to be able to consume data generated by open source Prometheus monitoring tools as well as both Telegraf and Open Telemetry agent software for collecting and correlating traces with relevant logs and metrics. Dashboards and topology and entity models also have been added to create more intuitive workflows and cross-linking among troubleshooting playbooks.

    The company also announced the Sumo Logic Multi-cloud Observability service, which provides monitoring and troubleshooting capabilities across Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP) and hybrid cloud environments. In addition, it announced the general availability of Sumo Logic AWS Observability, a service that automatically ingests, collects, unifies and analyzes telemetry data from AWS services such as Application Load Balancer, Amazon Elastic Compute Cloud (EC2), Amazon Relational Database (RDS), AWS Lambda, Amazon DynamoDB and Amazon API Gateway. It is designed to detect anomalous events, including the timeline and scale of those anomalies, and then employs machine learning algorithms to surface the root cause of the issue.

    Sumo Logic is also making generally available a Kubernetes Observability service that automatically instruments and discovers topology and entities of microservices-based applications and automatically collects logs, metrics, traces and metadata to populate preconfigured dashboards and alerts. The company also launched a Sumo Logic Web and Edge Observability service to monitor and troubleshoot content delivery network (CDN) services from Akamai, CloudFlare and Fastly, as well as similar services provided by AWS, Microsoft Azure and GCP.

    Finally, Sumo Logic has enhanced its SIEM service to add support for chain, threshold and aggregation rule types and the ability to schedule searches of specific types of security events. The reporting and data ingest capabilities of the platform have been enhanced as well.

    Bruno Kurtic, vice president of product management and strategy for Sumo Logic, said observability takes traditional monitoring to the next level because as applications are instrumented, observability becomes a property of the application. That approach provides IT teams with the context required to resolve issues faster. Existing monitoring tools are isolated silos that generate data which IT teams then must correlate on their own to find the root cause of an issue. That approach results in time wasted that could otherwise be applied to more value-added tasks, such as developing applications faster, said Kurtic.

    Fresh off raising capital by becoming a public company, Sumo Logic is differentiating itself via a more unified approach to observability that provides deeper analytics, is more secure, and affordable through the usage of credits that IT teams and purchase and apply to any service as they see fit, Kurtic said.

    Of course, Sumo Logic is not the only IT vendor aiming to take IT monitoring to the next level. DevOps teams have no shortage of observability options. The challenge now is determining how best to act on that intelligence once it’s presented.

  • New Relic Makes Revamped APM Platform More Affordable

    New Relic Makes Revamped APM Platform More Affordable

    New Relic has revamped its application performance management (APM) platform as part of an effort to make observability more accessible and affordable.

    Bill Staples, chief product officer for New Relic, said New Relic One has been updated to provide a new user interface designed to make it easier to navigate and explore an observability platform that now monitors both applications and IT infrastructure.

    At the same time, New Relic has reduced costs by adding a free tier with no hidden costs or credit card required, he said. It provides one free license to 100GB of data every month. The first 100 million application transactions per month free as are the first 1,000 incident events per month.

    Initial pricing for the next level tier is at 25 cents per GB of ingest on a per-seat basis with applied intelligence priced per transaction or event.

    Staples said New Relic is further reducing costs by leveraging open source agent software to collect data, which along with the revamped New Relic One platform will make observability ubiquitous.

    The move to reduce observability costs coincides with the rise of microservices-based applications that by definition have many dependencies that need to be monitored continuously. Historically, many IT organizations have limited use of APM platforms to only their most mission-critical monolithic applications because of the costs involved. Given their inherent complexity, however, microservices-based applications require IT teams to employ an observability platform. They also drive organizations to embrace best DevOps practices, of which observability is a core tenet.

    Staples said it’s still early days in terms of the adoption of observability platforms, noting in light of their absence, most of the adoption of DevOps thus far could be described as a half-measure.

    To further that goal, Staples said New Relic One will also make it more cost-effective for IT organizations to embrace distributed tracing as an alternative to application sampling without having to incur massive storage costs. Distributed tracing monitors all requests between services with each segment recorded as a span. All the spans of a request are combined into a single distributed trace to provide a method to monitor an entire request.

    Competition among provides of APM platforms is now nothing short of fierce. New Relic today has a significant share of the market, but as demand for observability platforms continues to expand, it has more rivals than ever. In the wake of the economic downturn brought on by the COVID-19 pandemic, IT organizations are also pressuring vendors to reduce costs. The challenge IT organizations face is that while they are trying to reduce costs, they are also rolling out more cloud-native applications based on microservices that require deeper levels of observability.

    It’s unclear how the battle of the future of observability will play out. The good news, however, is the cost of proactively monitoring IT environments appears to be dropping by the day.

  • Fastly Facilitates Observability at the Edge

    Fastly Facilitates Observability at the Edge

    Fastly this week announced it is adding the ability to collect tracing, customizable logging data and both real-time and historical metrics from a serverless computing framework designed to be deployed on edge computing platforms.

    Company CTO Tyler McMullen said the goal is to make it easier for IT to bring observability within the context of a DevOps to an edge computing platform.

    The Compute@Edge serverless compute platform enables IT teams to employ functions to execute code on an edge computing platform versus having to install virtual machines or containers. Those edge computing platforms are connected to a content delivery network (CDN) managed by Fastly.

    McMullen said that approach provides a way to deploy microsecond applications at the edge while still attaining observability in keeping with best DevOps practices. Otherwise, edge computing platforms become yet another black box that provides IT teams with no visibility.

    DevOps teams can send real-time logs to 27 and counting supported endpoints. Default log fields are automatically exposed to better determine the root cause of any issue. DevOps teams can also capture custom event details.

    Metrics such as CPU and RAM utilization can also be surfaced either in real-time or on a historical basis. The Compute@Edge service also honors request tracing parameters by maintaining them when they enter and leave the Fastly platform. Developers can tag individual end user requests with unique identifiers that can be shared with third-party platforms such as Datadog.

    Currently available in beta, Compute@Edge makes it easier for IT teams to employ a serverless computing framework to drive event-driven applications at the edge, McMullen said. Latency issues that arise when applications attempt to access cloud services are pushing more organizations to deploy real-time applications at the network edge. Functions running on a local serverless computing platform enable IT teams to achieve that goal in a way that consumes the smallest amount of IT infrastructure possible, he added.

    With the rise of 5G networking services, it’s now only a matter of time before the number of platforms on which a DevOps team needs to deploy code expands significantly. Most DevOps teams are already challenged by all the unique platforms to which need to deliver code. Edge computing could potentially exacerbate those issues because each edge computing platform tends to have unique attributes. A common serverless computing framework running on each of those edge computing platforms could help streamline continuous delivery.

    Of course, it’s too early to say what impact the downturn on the economy will have on edge computing projects. In theory, at least, organizations are gearing up to deploy these applications in anticipation of wireless 5G connectivity services being widely available and affordable. Chances are high, however, with more organizations not willing to send people to install applications on those endpoints manually, reliance on DevOps processes and CDNs to automate application delivery at the edge will become a requirement.

  • Grafana Labs Update Aims to Simplify Observability

    Grafana Labs Update Aims to Simplify Observability

    Grafana Labs today released an update to the open source Grafana project that adds a viewer that simplifies tracking the path of a single request through a distributed system.

    Version 7.0 also adds a plug-in for Amazon CloudWatch Logs along with support for Apache Arrow, a unified data and plugin framework that promises to make it easier to build plug-ins for the analytics and monitoring tools that are widely employed in DevOps environments to provide observability.

    The latest release also adds a range of automatic rendering options for a variety of formats, including tables and non-time-series charts alongside panel plugins for maps, pie charts and gauges.

    Ryan McKinley, vice president of applications for Grafana Labs, said Grafana has gained traction in DevOps environments in part because it makes it easier to query and analyze multiple types of data without having to pull it all into a central repository. All queries are launched against data wherever it resides, he said, noting by chaining a set of simple point-and-click transformations, users will be able to join, filter, rename and calculate all in the same platform.

    That approach makes it easier for DevOps teams to set up observability platforms using any mix of tools and services without much help from internal IT operations teams. That approach is now also starting to gain traction outside of DevOps environments as business analysts look for easier ways to also analyze data from different classes of applications, McKinley added.

    While observability has always been considered a core tenet of DevOps, achieving it has been challenging for most organizations. Simply setting up all the underlying platforms required has proven too challenging for many IT teams. Grafana has emerged as a tool that enables DevOps teams to visualize relevant data within the context of a custom-built observability platform.

    It’s not clear to what degree Grafana might emerge as a de facto standard. However, during an economic downturn, it’s not uncommon for IT organizations to rely more on open source tools to reduce costs. Rather than having to lay off an analyst or developer, many organizations will first look to reduce their reliance on commercial software licenses. Many of those same organizations have been wrestling for years with a fragmented approach to analytics software that often can yield conflicting insights.

    McKinley said one of the benefits of Grafana is that it reduces the cost of developing different models that enable organizations to explore different potential scenarios. Too often organizations are wedded to specific models that might not reliably reflect a new business trend simply because the cost of creating a new model is too high or takes too much time.

    Whatever the analytics path forward, the one thing that is clear is that as IT continues to become more complex thanks in part to the rise of microservices, DevOps teams will no longer be able to function without some form of observability platform being readily accessible. The only real question now is how best to go about building and maintaining that observability platform.