devopsy.kinsta.cloud

Tag: observability

  • Optimizing Prometheus Implementations With Managed Services

    Optimizing Prometheus Implementations With Managed Services

    Prometheus is a leading metrics-based open source monitoring solution and time series database used across the IT community to collect metrics from systems, applications and services. According to the 2021 Cloud Native Survey from the Cloud Native Computing Foundation (CNCF), the use of Prometheus in production has reached 65%, an increase of 43% year-over-year.

    “As cloud-native stacks have grown and matured, observability has become crucial for visibility into application metrics, performance, alerting, and insights,” said Chris Aniszczyk, CTO of CNCF. “Prometheus is a key element of this, enabling organizations to build a foundation of observability.”

    If you are using Prometheus, it is probably because of its many advantages including reliability, easy initial implementation and its open source nature. However, in addition to these benefits, Prometheus also brings several challenges if you are deploying and maintaining the solution on your own.

    Facing the Four Main Prometheus Challenges

    Prometheus users face four primary challenges that can be alleviated by the right service provider:

    Challenge One: Support

    Prometheus users face the dilemma posed by any open source tool: There is no vendor behind the solution to deliver technical support. It is easy to take tech support for granted, but when you are working with a technology that is not supported, even a small issue can become a substantial problem. A commercially-backed product can provide a critical level of expertise you do not have in-house, and take on responsibility for keeping the system up and running.

    Challenge Two: Maintenance

    The greatest challenge you’ll likely face with Prometheus is the amount of time your team must spend configuring and maintaining the system. 

    Just one example of how Prometheus drains your resources is the use of third-party exporters to expose metrics in the Prometheus format. These integrations are designed by members of the open source community an do not come with any support; upgrades are hit-or-miss.

    Exporters are difficult to maintain. First, there are often multiple exporter options and choosing the right one for each of your services is time-consuming. Second, you can easily have hundreds of different services running in your environment and your team can spend hours every day making sure the exporters are working properly.  

    Another maintenance issue involves PromQL, the Prometheus query language. PromQL is a custom query language, and it is very complex and difficult to learn. Having to rely on your one PromQL expert for all Prometheus queries across the organization is simply not practical or sustainable.

    With all these maintenance requirements, you will have some team members spending all their time managing and maintaining a Prometheus environment, eating away at resources that could be used to focus on business needs and innovate new applications.

    Challenge Three: Scalability

    Users tend to deploy Prometheus in silos, with a separate server for each cluster. Over time, you may have tens or even hundreds of Prometheus servers in your environment that you are not aware of. This leads to scalability issues regarding consistency, maintainability and a global view of your metrics.

    In addition, storage is a major hindrance to scaling Prometheus, because the system typically stores metrics for about two weeks. If you need to analyze metrics older than a couple of weeks, you run into a serious problem. 

    Challenge Four: Correlating Data

    Data does not have much value if you cannot correlate it and, unfortunately, Prometheus does not provide correlation for metrics to other elements in your environment like processes or cloud regions. This makes it difficult to pinpoint the source or impact of a problem. 

    Options for Solving Prometheus Challenges

    Options are available to help you with Prometheus but most do not solve the four major issues outlined above. The following are your options:

    APM: APM vendors all have some level of support for Prometheus, but this is an add-on to their platform. They offer an ingest tool that can transform Prometheus data into a format you can query natively within the APM platform. In reality, they are not helping you utilize Prometheus but instead are forcing you to work within the constraints of their system.

    Cloud tools: Some cloud hyperscalers offer solutions to support Prometheus. These tools can help with deployment and infrastructure maintenance, but you are still stuck doing all the other configuration and maintenance work. 

    Managed service: The preferred option is a Prometheus managed service, which can alleviate all four of the main challenges posed by open source Prometheus. The right optimized managed service provider delivers support, maintenance, long-term storage and data correlation.

    Must-Have Capabilities for Your Prometheus Managed Service Provider

    The following are essential capabilities you should look for when choosing a Prometheus managed service provider:

    Prometheus compatibility: You need an optimized managed service provider whose platform is designed to be natively compatible with Prometheus, consuming data in the Prometheus format and allowing you to run PromQL queries on the data.

    Simplified PromQL: To eliminate the reliance on PromQL experts for every query, a managed service provider should simplify the use of PromQL, without breaking compatibility.

    Exporter management: The right optimized managed service provider will be able to streamline the selection, deployment and management of exporters, saving you a significant amount of time and effort.

    Automated discovery: Tracking and optimizing all your services can be challenging. For this reason, a Prometheus-managed service should be able to automatically discover the services running in your environment and help guide you with configuration.

    Metric Storage: One of the shortcomings of Prometheus is the lack of long-term storage. Consequently, a Prometheus managed service provider should provide a database for long-term storage of metrics, so you can perform comparative analyses of the data, from month to month and even year-to-year.

    Prometheus compatibility: If you already use open source Prometheus, your teams are used to using things like PromQL and Grafana dashboards. You should select a managed service provider that is Prometheus-compatible so your team can use the same dashboards and queries they have always used to view the data.  

    Conclusion: Optimized Valuable Tool

    Prometheus is not a standalone solution. But it is an optimized valuable tool to collect metrics for performance analysis and it is appreciated by your architects, developers and IT Ops pros. While Prometheus may have some limitations, it is worth finding a solution to these issues; that solution is a managed Prometheus service. The right managed service provider can give you the best of both worlds—all the benefits of Prometheus without the time, resources and expense needed to run it effectively.

  • Embrace Change in Production to Improve MTTR

    Embrace Change in Production to Improve MTTR

    DevOps creates value through constant deployment changes. When done right, more frequent deployments allow teams to continuously improve their products or services while making it easier to manage the effects of the change. One way to do this is with continuous integration and continuous deployment (CI/CD). CI/CD helps to reduce the risks to each build and allows teams to release valuable features quickly and safely. But CI/CD alone is not sufficient for understanding change and service impact in production. The recent DEJ State of IT Performance Report found that change is the largest source of production issues. A full 76% of all performance problems can eventually be traced back to changes in the environment but 67% of organizations don’t have the ability to identify changes in their environment that caused performance issues.

    In a modern IT environment, different teams are constantly integrating new features and new technologies and renewing their stack. With many moving parts, it’s impossible to mimic the production environment when testing new features in pre-production. By the time you’ve implemented and tested new code, your branch will be different from what is in production.

    Here’s the reality we’re working in:

    ● Integrating testing in CI/CD pipelines will not keep bugs out of production. Some organizations don’t do true CI/CD, meaning some processes are not even tested and automated in the pipeline.
    ● User configuration errors are common. Here’s a consolidated list of configuration errors that brought down production.
    ● Change anxiety is real: Ops folks get nervous when hitting the deploy button, even for small changes.

    Changes are inevitable in DevOps, but organizations can protect themselves from a performance incident or an outage by integrating change in production safely with a few simple steps.

    Build Data Connections

    Today, data is kept in silos, which is why it’s time-consuming and challenging to make associations and leverage them to find the root causes of performance issues. DevOps teams need to connect their data in meaningful ways to troubleshoot complex issues in distributed systems. Building the “connections” foundation is critical to understanding the state of a system and how the pieces interconnect and interact.

    How well an application performs in production is related to a combination of code, infrastructure and operations. For instance, when an incident occurs, teams should quickly know if that issue is related to code, infrastructure or operational activity. Understanding these connections and correlating the data cannot be performed manually. We have hit the wall in linking together data silos; we just don’t scale as software does. Teams need to consider a monitoring and analytics approach that connects operational data together from the start and models the causal links between the data for real-time root cause diagnosis.

    Understand Change Impact to Avoid Outages

    How well can your DevOps team identify the key changes made to your production environment? If your answer is “not very well,” then you’re missing critical data that helps to speed up troubleshooting. The lack of visibility into changes and how they impact production makes it impossible for operations staff to identify the cause—the very thing that SREs care most about.

    Changes are made to the code or the cloud infrastructure, either directly or through infrastructure-as-code (IaC) templates, for a variety of reasons. The risk of misconfiguration and performance issues increases when these operational activities are not continuously monitored and correlated across the stack.

    Metrics don’t tell the whole story. Logs don’t pinpoint where to act. Simply collecting these data types leaves many companies struggling with knowing what is important and what’s not when it comes to real-time troubleshooting.

    Having a shared “system of record” for changes for all the different streams of activities increases visibility and control of all changes in real-time. Teams can see pre-incidents and be aware of increased risks when changes are made, especially during high-velocity change periods. By seeing how a code change can impact things down the line, developers can proactively troubleshoot and debug code.

    Implement Real-Time Root Cause Diagnosis

    In the same DEJ report, organizations shared that 66% of MTTR is spent on the ‘diagnose’ stage. This correlates to the 67% of teams that are unable to identify the changes in their environment that caused performance issues.

    With a shared “system of record” for changes, DevOps teams can troubleshoot incidents in real-time, dramatically cutting the time spent on the “diagnose” stage of MTTR. This is especially effective when teams can see all the changes and other events that are happening to the system in a single, searchable/filterable real-time feed; this timeline would provide the potential causes of issues.

    The best tool would allow you to start from an effect and work backward, chaining multiple events overlaid with observability data to determine the causes quickly. Building the causal chain from the end helps teams hone in on the connected change and answer each “why” in a straightforward way. This capability makes it easier and faster to prioritize relevant changes and resolve performance degradations or outages with actionable context.

    Communicate Everything

    During diagnosis and resolution, it’s important to see what’s being done without having to ask on Slack, wait for each other or unknowingly interfere with remediation actions. The best option would be leveraging the same timeline to capture all the context of an incident at the moment it occurs, and at any moment in time during diagnosis and resolution, to avoid reverting each other’s changes.

    While an incident postmortem brings teams together to gather data and discuss the details and learnings from it, this process is often inefficient and time-consuming. Each team member involved in the incident response effort would have to recollect what they observed, what actions they took and what they communicated to other team members. What if a system could capture all the actions and communications during an incident response effort? Wouldn’t it be nice to just summarize your thoughts in one sentence and be done with the reporting?

    When preparing for planned and unplanned changes that can impact the bottom line, DevOps teams need to consider these steps to gain true observability of their operations. Focus on finding the connections in the data to explain cause and effect. Understand change impact —as those changes are being made —to avoid outages. Capture and communicate activities as seamlessly as possible and identify and capitalize on opportunities to automate the workflow.

  • New Relic Survey Unearths Observability Challenges

    New Relic Survey Unearths Observability Challenges

    A New Relic survey found only 27% of respondents have achieved full-stack observability with only 5% claiming they have a mature observability practice in place. A third (33%) of respondents also said they still primarily detect outages manually or based on complaints, the survey found.

    The global survey polled 1,614 respondents, including 1,044 practitioners and 570 IT decision-makers and was published today by New Relic.

    On the plus side, the survey also found nearly three-quarters of respondents said C-suite executives in their organization are advocates of observability, and more than three-quarters of respondents (78%) saw observability as a key enabler for achieving core business goals.

    However, more than half (52%) of respondents said they experienced high-business-impact outages once per week or more and 29% said they take more than an hour to resolve those outages.

    Peter Pezaris, senior vice president for strategy and user experience at New Relic, said a big reason it takes so long to resolve issues is the time required to determine the root cause of issues using multiple tools. A full 82% of respondents said they use four or more tools to monitor the health of their systems. Only 7% said their telemetry data is entirely unified and only 13% said the visualization or dashboarding of that data is entirely unified. Nearly half of respondents (47%) said they would prefer to have a single, consolidated observability platform, but only 2% said they currently used one tool for observability.

    New Relic, of course, is making a case for a single observability platform that spans both the application environment and the underlying infrastructure an application depends on. Today, most IT teams are using a wide range of tools to monitor infrastructure and applications.

    Nearly all respondents said by 2025 they expect to deploy capabilities like network monitoring, security monitoring and log management, as well as Kubernetes monitoring. More than a third (36%) believe observability increases their productivity and enables them to find and resolve issues faster. Just under a third (32%) said observability enables cross-team collaboration, while more than a quarter said observability improves their ability to innovate.

    Nearly three-quarters (72%) expected to maintain or increase their observability budgets next year. More than half (52%) of respondents, including 57% of C-suite executives, expected observability budgets to increase over the next year. A total of 14% of all respondents and 16% of IT decision-makers expected to increase observability budgets significantly or extensively. IT decision-makers anticipate needing observability most for artificial intelligence (51%), internet of things (48%), edge computing (38%) and blockchain (36%) in the next three years.

    It’s still early days as far as the adoption of observability platforms is concerned, but it’s apparent that as application environments become more complex the plethora of monitoring tools that DevOps teams rely on will need to be streamlined. Observability platforms promise to unify logs, metrics and traces in a way that makes it simpler to launch queries to identify the root cause of an issue.

    The rate at which DevOps teams will embrace observability will naturally vary, but the biggest obstacle might not be the platforms themselves. Instead, the issue is understanding what queries can help DevOps teams better understand the root cause of an IT issue before there is a major disruption. In the long term, it’s expected that machine learning algorithms will leverage the data collected by observability platforms to automatically identify issues that might lead to a disruption long before it actually occurs.

    In the meantime, the hope is that monitoring tools will give way to observability platforms that will make managing complex IT environments much simpler for all concerned.

  • Lightstep Extending Beyond Observability with Lightstep Incident Response – Techstrong TV

    Lightstep Extending Beyond Observability with Lightstep Incident Response – Techstrong TV

    Ben Sigelman and RJ Jainendra discuss ServiceNow’s latest announcement that Lightstep is extending beyond observability and creating a differentiated portfolio for app development with the general availability of Lightstep Incident Response, helping make organizations’ digital products and services more reliable and resilient. The video is below followed by a transcript of the conversation.

    Alan Shimel: Hey, everyone. Welcome to Techstrong TV’s segment today. We’ve got some breaking news I wanna bring to you, and I’m really happy to be joined by two gentlemen who are gonna tell you the news. I’ve had the pleasure of having them on before.

    They’re two really bright guys from ServiceNow. Let me introduce you to Ben Sigelman and RJ Jainendra from ServiceNow. RJ, I think I left out an “N” somewhere in that second or third syllable, but we’ll let you correct it. I’m gonna ask you to go first, RJ. Introduce yourself to the audience.

    RJ Jainendra: Hey, Alan, good to see you again. And no, you got it right, RJ Jainendra. I’m a VP and GM at ServiceNow, unlocking product growth at the company. It’s great to be here.

    Alan Shimel: Excellent. It’s good to see you again. And of course, RJ, you’re general manager, but you’re the portfolio you are GM with includes observability. I believe it includes value stream as well, yeah?

    RJ Jainendra: So, ServiceNow does have offerings for value stream management, which brings together a number of different product lines. Our IT business management, which we recently rebranded as Service Portfolio Management, SPM, that’s one. The second is it also brings in elements from devops and ITSM offerings to provide a value stream management.

    But I’m here today to talk about Lightstep Incident Response. That’s the product line that we’ve been building out for a little while now, and ready to launch here today.

    Alan Shimel: Absolutely, so, we’re gonna talk about Lightstep. Gotta have Ben here. Ben, welcome. Why don’t you give people a little bit of – give the folks a little bit of your background?

    Ben Sigelman: Yeah, thanks, Alan, for having us. So, I’m Ben Sigelman. I was one of the cofounders at Lightstep and was the CEO up through last year, when we were acquired. ServiceNow wasn’t willing to let me remain the CEO of ServiceNow after the acquisition, so now I’m also a VP and GM, along with RJ. 

    Lightstep’s brand and certainly our products and many aspects of our culture have actually been intentionally retained as part of the acquisition, and the brand has actually been expanded to be a multi-product brand, which is what we’re talking about today.

    Most of my day-to-day is spent doing a combination of thinking about the product vision for observability specifically within Lightstep as well as the longer-term strategy within ServiceNow, and what we can do for our customers by unifying the world of ServiceNow with the world of high-volume customer-facing applications that Lightstep tends to play in.

    Alan Shimel: Absolutely. I think the ServiceNow team already has a pretty good guy in its CEO, but hey.

    Ben Sigelman: I actually, when I talked to Bill during the acquisition, I told him he had my condolences for being a CEO of a much larger company. I knew how stressful it was for Lightstep, so I can only imagine what it’s like for him. But he’s a fantastic guy.

    Alan Shimel: Absolutely. Yes, until you’ve been a CEO, and I don’t care whether it’s a really, really big company or a really small company, yeah, it’s a lot. Anyway, guys, let’s talk Lightstep, observability. Let’s talk about the expansion. Just to level-set things – you both referenced it – Lightstep was one of the leaders in this observability revolution or wave that we saw overtake the industry, let’s say, in the last three, four years, certainly, it really caught fire.

    Lightstep, Ben and your two cofounders, I believe, all came from Google, and really in many ways helped initiate the whole observability revolution there. Then leveraging open source and working with open source, open telemetry, et cetera, really built quite a community and following.

    ServiceNow, one of the most dynamic companies in our industry, recognized this and made Lightstep and observability an important part of the ServiceNow offering. But from what I’m hearing and reading between the lines is why stop there, right? There’s more.

    There’s more that the marriage of ServiceNow’s know-how market intelligence and skill set, and the Lightstep platform can do more, and today marks a significant expansion of that. So, you’re both GMs. Who wants to go first to talk about what this new expansion is?

    RJ Jainendra: So, I can talk a little bit about what the expansion is, and Ben can, I think talk through why it makes a ton of sense, what we’re doing. The thing that we see that’s happening in the market is just the amount of investment going into what’s been called digital transformation.

    It’s just increasing significantly, and I think by some accounts, IDC forecast, it’s gonna be something like north of $10 trillion cumulative by 2025. So what Marc Andreessen said a long time ago, I think it has come to pass, which is every company is a software company, and increasingly, with the adoption of cloud, organizations are prioritizing cloud operations and software development teams they build, and they run these millions of new customer-facing, revenue-generating applications and services.

    So from that perspective, the SREs and developers who really are responsible for keeping applications reliable and resilient, they really start impacting the business outcome. That’s where if you have incidents that occur, maybe it’s a bug or some sort of an outage or poor performance, it has a major impact on revenue, on customer loyalty, on the brand perception.

    That’s why you see the option in the market to introduce a new offering around incident response, which really leverages the power of the ServiceNow platform, but then it brings together the observability capabilities that Lightstep has into sort of a holistic offering that we can bring into market to help customers become more proactive and more effective in addressing issues. Ben, do you wanna add to that in terms of your perspective from an observability side?

    Ben Sigelman: Yeah, for sure. I think the way I’d explain it is observability, Alan, I would agree with you that it’s the wave, but it’s a very confusing, diffuse – it’s a body of water right now that can be hard to wrap your head around, and you frankly have a lot of what I would call legacy observability companies trying to take what they have and call it observability, because that’s easier than reforming it.

    So, it might be helpful to spend just one very quick, 30 second snippet talking about what observability actually is, architecturally. You have some layer at the bottom of integrations and data acquisition. You mentioned Open Telemetry earlier. The Lightstep Incident Response product that we’re announcing today also has an integration ecosystem built around it.

    But you have the acquisition of data, you have some kind of data platform where you store that, which I’m not gonna talk about in detail right now but I think is an interesting topic, slightly far afield. Then you have workflow been made with your full knowledge and workflow built on top of that, and that’s where the actual value is delivered.

    A lot of the conversation around observability today is that you have a silo for your metric system, a silo for your logging system. Maybe you tack on some kind of tracing or PM system, and then you integrate them in the browser.

    Then you also have this Incident Response thing over there. That’s some other vendor, and it does not make any sense. When we talked with our customers, the three things they are prioritizing are overall performance, velocity, and actually, most importantly, incident response and timed resolution.

    It’s the most important thing for observability. Then you have not just a separate tab, but a separate product to do incident management and incident response, which is not sensible from a workflow standpoint.

    So, for the brand, this is a very obvious and natural thing, to have incident response and observability integrated into a single experience, especially incident response where seconds matter, and a context switch like that is very expensive. So, anyway, that’s why this makes a ton of sense to us.

    It goes right at the heart of the MPTR problem, to have a single platform to manage the full lifecycle of incident management on the observability side and on the incident response side.

    Alan Shimel: Absolutely, and you know, Ben, to your point – well, first of all, I think you call them “legacy observability vendors.” Really, they’re APM, application performance management vendors who – 

    Ben Sigelman: Or infrastructure monitoring vendors, more to the point.

    Alan Shimel: Right, or infrastructure – 

    Ben Sigelman: It’s not the –

    Alan Shimel: You know, everyone loves a good name.

    Ben Sigelman: Right.

    Alan Shimel: A company I had started, we had come out with a product that Gartner would later name the category NAC, network access control. We had some stupid name for it before, but Gartner, you know, did that.

    All of a sudden, out of the woodwork, all these people who were doing totally different things started calling what they do NAC. A very similar thing happened with observability, right?

    Ben Sigelman: Yeah.

    Alan Shimel: Anyone who’s doing monitoring, all of a sudden it was observability. But I think, also to your point, was that observability got siloed into just observing. Almost by definition, observing, observability, is not actionable. It’s almost like observability is a passive word, where actionable is active word.

    Ben Sigelman: You’re exactly – that’s a very good point. If you go – I tend to be a little academic, but I think this is an interesting point. Observability really originates from control theory in the 1960s, and it was this smaller, kind of lesser cousin to controllability, which is kind of what you’re talking about.

    Controllability is really the thing that matters. If you can control a system, there are scientific ways of talking about that. Mathematical ways talking about that. To a certain extent, you can think of incident response as being the controllability side of what we’re talking about here.

    It’s like we’re making insights from observability actionable with the Incidence Response product, and that’s a really important part of actually closing the loop on the control theory of modern distributed systems. That’s a great insight that you’re making, Alan.

    Alan Shimel: Yep, thank you. Every once in a while, even a blind squirrel finds something, right? So, RJ, let’s talk about this. ServiceNow is not a trivial size company. To decide that – you didn’t really have an incident response solution in market prior to this, as far as I know. I’m not an expert on everything ServiceNow, but I’m not aware of it.

    To say okay, we are going to build this in as the logical next step in the Lightstep platform, just the inertia sometimes in an organization your size, it’s difficult. Let’s hear from your point of view what it took to make that happen, turn that ship.

    RJ Jainendra: Yeah, so, Alan, we actually do have capabilities in the ServiceNow platform through our IT Service Management and IT Operation Management products, around things like event technology around event correlation, being able to take large-scale number of events from the observability tools and bring them in and understand what those events mean.

    We also do have functionality in ITSM around notifying the right person. But really, the ServiceNow technology, we have great adoption with IT organizations and a large number of customers. But what we’re seeing really is as our customers are going through the digital transformation, they’re reorganizing, and the product teams that are out there are the ones that are sent to adopt these technologies on observability. 

    That’s why we felt that we wanted to bring an offering into market that could be adopted by teams. So, the entire experience for Lightstep Incident Response is as a modern SaaS application. If you’re a developer on the team – and what we’re finding right now is a lot of our initial customers are smaller companies.

    They’re early growth-stage companies that are post-revenue. They have customers, they wanna make sure that their systems are highly available for them. So, what we’re finding is that they can come online, they can sign up, and within a few seconds get access to the system, start using it, and if they see value, then they can swipe the credit card and they’re off to the races.

    So, very much a different motion, really suited to how teams out there in smaller companies are adopting technology. And of course, the startups of today are the ones that become the unique ones tomorrow, so that’s what we’re doing that’s a little bit different. But for ServiceNow customers, they need to have those capabilities within the ServiceNow platform.

    Alan Shimel: Excellent. So, guys, let’s talk – if you don’t mind, I gotta do a little meat-and-potatoes here. You guys announced this today. Is it available? When will it be available?

    RJ Jainendra: It is available today on Lightstep.com. So, we have had the product in the market for a little bit. We did a soft launch earlier last year. So, it’s been battle-tested, and we have a number of customers that have come online. They’re using it in production. So, it is available today, and it doesn’t take more than an email to sign up and get going.

    Alan Shimel: What’s the offering? How is it priced? If you’re an existing Lightstep or ServiceNow customer, how do you – firing off an email, yeah, but how do I – what’s this gonna set – you know, what’s involved here?

    RJ Jainendra: Yep, so, actually, it isn’t a whole lot more complicated than what I just said, which is just as you sign up for a lot of consumer-type services online, developers and SREs can get started with an email. We do have a couple of different tiers. So, we do have a free tier, where if you are just getting started and you may have just a couple of services that you’re monitoring and addressing issues for with Incident Management, that’s free.

    Now, if you’re gonna start scaling up and you have more than two, that’s when the pricing kicks in. We are taking an innovative approach in terms of pricing, which is it’s based on usage. So, we look at the number of active services that a team has, and then we charge by the number of active services, which is very different than some of the other solutions in the market, which charge by the number of developers that you have in the system.

    We really think that we wanna align to the value that our customers are trying to deliver, which at the end of the day, if you’re a developer or if you’re an SRE, your goal is to deliver these resilient, reliable services. So, we feel that aligning to the services – and no matter whether you wanna bring in your whole team or maybe you also wanna bring in sort of extended folks into the solution to be able to collaborate and figure out what the problem is.

    So, we don’t wanna let the number of seats become an impediment. We really would like to have our customers drive this notion of service ownership across the entire team, as opposed to being owned by a few key people, because there’s a lot of burnout that happens.

    So, there’s different models, but after a 30-day free trial, if customers see value and they have these active services, that’s when the billing kicks in. It’s all done by credit card, so it’s a fairly streamlined experience.

    Ben Sigelman: I will add that Lightstep observability experimented with seat-pricing several years ago and abandoned it, because it’s very important for us to deliver value to our customers, and there’s nothing quite as frustrating as hearing people say, oh, we really wanna be using this, but we don’t wanna add another seat license and blah, blah, blah.

    I think especially in large organizations, for incident response, it’s just immensely frustrating to be kind of wanting to use something, like, sporadically, maybe once every couple of months, and then being blocked out of the application because of concerns over seat licensing.

    So, with Lightstep Incident Response, there’s really no restriction on that. You can have a thousand-head engineering organization and put them into the product, and that doesn’t change your bill at all. The pricing is more correlated to the value that we believe we’re delivering, and the resiliency of the services themselves.

    Alan Shimel: Sure. So, I’ve gotta ask the question. I’m gonna ask it of you, Ben. This is a step of taking observability down the control, as you mentioned, the control path, for full control. What’s next?

    Ben Sigelman: That’s a good question. We’re gonna move into crypto. No, I’m just kidding, we’re not.

    RJ Jainendra: Join the crowd, yeah.

    Ben Sigelman: I’m not here to talk about specific products, but directionally for Lightstep as a brand, I think that we see there’s an entire ecosystem out there of – I think you can kind of extrapolate from where I was going with the workflows.

    I think where we – actually, Lightstep and ServiceNow are incredibly similar in our belief in workflows being paramount, and there are a number of other workflows where people have to do a major context switch when they’re involved with the work of making reliability and velocity happen for cloud-native applications and for customer-facing applications. That’s kind of a hint as to, I think, where we’re headed with future products. But this certainly isn’t the last one yeah.

    Alan Shimel: Great. Guys, we’re way past 15 minutes, but for people who wanna go get Lightstep and Lightstep observability up and running today, where do they go?

    RJ Jainendra: www.Lightstep.com.

    Alan Shimel: That’s what I was looking for, RJ. Hey, gentlemen – Ben, RJ – thanks so much for coming on and breaking this exciting news and offering, new offering from the Lightstep/ServiceNow team. Keep us posted, best of luck with it. It sounds like a winner. We hope to hear more soon.

    RJ Jainendra: Thanks, Alan.

    Ben Sigelman: Thank you, Alan.

    Alan Shimel: Thank you, guys. All right, go check it out, Lightstep.com, their new observability and incident response now as part of it, from ServiceNow. We’re gonna take a break. We’ll be right back.

  • Time-Series Database Basics

    Time-Series Database Basics

    There’s been a lot of buzz about time-series data for the past few years. No matter what you’re monitoring—financial data, server performance, social media activity or something else entirely—time-series data can give you valuable insights into trends and patterns.

    But what exactly is a time-series database, and why do you need one? This article will look closely at time-series databases and how they can help you make sense of your data.

    What is a Time-Series Database?

    A time-series database (TSDB) is a database management system optimized for storing and querying data that changes over time. Time-series data is often used in monitoring applications where it’s important to quickly retrieve information about a system’s current state and trends and patterns over time.

    In most definitions, time-series data, often referred to as time-stamped data, is a sequence of values indexed in time order. Time stamping refers to data collected at various times, where each value is time stamped. These data points are usually gathered from the same source and are used to measure progress over time.

    Important Notes:

    You can use relational or NoSQL databases to crunch time-series data, but purpose-built time-series databases are tailored to exploit the unique features of time-series data. This implies that time-series databases ingest at a faster rate, query more quickly and compress data more efficiently. Furthermore, time-series databases include special analytical capabilities and management features that are not found in most relational or NoSQL databases.

    Some Advantages

    This list is not exhaustive, but here are some of the advantages that you might get from using a time-series database:

    1.) Time-series databases can handle high-velocity data very well.

    2.) Time-series databases are purpose-built for storing time-series data, making them more efficient in storage and querying.

    3.) Time-series databases often have built-in analytics and management features designed specifically for time-series data.

    4.) Many time-series databases are open source, which means they’re free to use.

    Some Disadvantages

    There are also some disadvantages that you should be aware of before using a time-series database:

    1.) Time-series databases are often more complex to set up and manage than relational or NoSQL databases.

    2.) There are relatively few time-series databases to choose from, so you might not have as much flexibility in choosing a platform that meets your specific needs.

    3.) Time-series data can be very large, so you’ll need enough storage capacity to accommodate your data.

    What Are the Characteristics of Time-Series Data?

    There are several characteristics of time-series data that make it unique and require special handling. From a database standpoint, the most essential ones are as follows:

    1. Timestamp: Every data point has its timestamp. The timestamp is crucial for calculating or analyzing the information.
    2. Structure: Metrics from devices or monitoring are almost always structured, unlike those produced by Internet applications. They have predetermined data types and lengths, and the structure will not alter until the device’s firmware is updated.
    3. Stream-like: Data sources, such as audio or video programs generate data at a set or constant rate. These data streams are completely unrelated to one another.
    4. Stable flow: Time-series data traffic is constant over time, and it may be calculated and predicted if enough data sources are used within a certain sampling period.
    5. Immutability: The source of this data is a time-series data store. Each data point is created only once and never updated or corrected. Time-series data, like log data, is typically append-only.

    How Is Time-Series Data Used?

    Many of you may be wondering why time-series data is so important. The answer is that it allows us to track changes over time, which can be incredibly valuable for monitoring and troubleshooting purposes.

    For example, let’s say you’re a system administrator tasked with keeping an eye on server performance. By monitoring various metrics—such as CPU usage, memory usage and network traffic—you can quickly identify when a server is starting to experience issues.

    Time-series data can also be used for predictive purposes. For example, if you notice that CPU usage spikes at certain times of day, you can use that information to plan for future capacity needs.

    Of course, time-series data isn’t just limited to server performance. It can be used for any application where it’s important to track changes over time. This includes everything from weather and financial data to social media and website analytics.

    Let’s Get Specific

    Here’s what you should know about time-series data: It’s typically used to seek insights into operations, create alerts based on real-time analysis and forecast future trends.

    The following characteristics are found in time-series data applications:

    1. High write-read ratio: Twitter and LinkedIn are internet applications with single articles that millions of people read, but raw time-series data is primarily scanned and evaluated by apps and algorithms.
    2. Retention policy: In general, time-series data is not kept permanently. Organizations have a retention policy that specifies when and how their data is destroyed.
    3. Real-time analytics and computing: Time-series data must be calculated in real-time to identify out-of-the-ordinary activity and sound alarms based on the acquired information or aggregate findings.
    4. Query scope: Time-series data is generally requested over a period or a set of data sources, and filters are used to prevent all historical information from being requested. Furthermore, all or a portion of the data sources with a filter condition are always aggregated.
    5. Trends: Single data points are typically not significant in time-series data. The emphasis is on how data evolves, such as fluctuations in the previous hour or day.

    Popular time-series solutions like TDengine or InfluxDB, for example, create more efficient processing of time-series data and greater performance than general databases by taking advantage of these qualities.

    Do Time-Series Databases Need Specialized Databases?

    We live in a modern world where data is constantly being generated at an unprecedented rate. To keep up with the demand, businesses need to be able to store and process large amounts of data quickly and efficiently. This is where specialized time-series databases come in.

    Everything is online, including meters, automobiles, lifts, assembly lines and even bicycles. And these items are sending out a never-ending stream of numbers and events that IoT and the cloud have unleashed an explosion in time-series data generation.

    With this in mind, it’s not surprising that several specialized time-series databases have been created to deal with this type of data influx.

    Time-series data sets are enormous and pose a significant problem for general database management systems, such as relational and NoSQL databases. Non-specialized databases have trouble with the following elements of time-series data:

    1. Data ingestion rate: In many time-series data scenarios, millions of data points are generated every second and must be ingested in real-time. Relational databases are not built to handle this quantity of data, and while NoSQL databases may be scaled to do so, the amount of resources required soon becomes prohibitive.
    2. Query latency: It is often the case that a time-series system must scan a massive number of data points to get an aggregate result, which might lead to sluggish performance. For example, it would take days for a basic database to compute the average response time of all Amazon.com clicks, by which time the overall conclusion may be incorrect.
    3. Storage cost: Data generated by internet-connected devices and apps is nonstop 24/7, with a single day producing as much as a terabyte of data. Because relational and NoSQL databases cannot compress this data effectively, storage costs can quickly skyrocket.

    These problems are generally linked to efficiency in processing big data sets, although there are some places where general databases frequently do not meet even the fundamental demands of time-series applications:

    1. Data life cycle management: Time-series data is generally removed in bulk, not one data point at a time, as it ages out.
    2. Roll-up: In most cases, time-series data is collected and rolled up over a set period before being stored in the new table. Raw data and rolled-up data can have distinct life cycles and retention policies.
    3. Special analytic functions: Time-series applications need more specialized features than general databases, such as time-weighted average, moving average, cumulative sum, rate of change, elapsed time for a specific state and the delta between two consecutive data points.
    4. Interpolation: The database management system must be able to interpolate data based on the adjacent data points and rules to regularize data sets when applications or algorithms require.
    5. Continuous query: Therefore, if you hear a user or customer say they want to know when their data is updated, it’s reasonable to assume they want this functionality.
    6. Session and state windows: Aggregations and analytical procedures may be performed on a session or state window, not just time—for example, consider one that only calculates average power consumption when a machine is switched on.

    With general databases, developers must write custom code to implement features specific to their data set. Different data workloads require different database solutions; one size does not fit all. For time-series data, no matter the size of your data set, a purpose-built time-series database is the best tool for the job.

    Why Are Time-Series Databases Becoming Popular?

    Two years ago, time-series databases were the most popular database type in the business, owing to its expanding number of use cases. It is especially useful if you’re executing sophisticated transactions like advertising, e-commerce, supply chain management and so on.

    However, because of the rapid expansion of the IoT, they are becoming even more popular. As more devices become internet-connected and send data—time-series, of course—to the cloud, many industries are interested in purpose-built time-series databases.

    Time-series data is proving to be an important tool, not just for decision-making and optimization but also for industrial applications. Finally, IT infrastructure has been growing rapidly, with everything from servers, containers, network devices, apps, and microservices being monitored to create massive quantities of time-series data.

    Older time-series databases, on the other hand, are often closed systems employing antiquated structures that do not scale to handle the increasing amount of data. A million time-series data points used to seem like a lot, but now millions and even billions of data points are commonplace.

    Finally, integrating old (previous) time-series solutions with popular data analysis tools like artificial intelligence and machine learning platforms is difficult, if not impossible. These legacy systems cannot be transferred to the cloud without a lot of work and their licensing terms are no longer adequate for current apps.

    Final Thoughts

    The expanding market and the constraints of previous time-series databases are making room for a new generation. These new time-series databases are built from the ground up to take advantage of the cloud, big data and AI/ML technologies.

    If your business or application deals with time-series data, you need a purpose-built time-series database. The advantages of using such a database are too great to ignore, and the disadvantages of using an older, less capable database are becoming more and more apparent.

  • Open Standards Are Key For Realizing Observability

    Open Standards Are Key For Realizing Observability

    Observability has quickly become a major focus of enterprise DevOps. Yet tool sprawl and complexity can hold some observability initiatives back. The 2022 CNCF Microsurvey on observability trends found that 23% of organizations use between 10 and 15 tools for monitoring, metrics and gathering logging and tracing data. A full 50% also noted that engineers and teams using multiple tools presented a top challenge to observability.

    I recently met with Dotan Horovits, principal developer advocate for Logz.io, to get his take on improving how modern development teams handle observability. According to Horovits, open standards and open source projects are the principal drivers to realizing interoperable observability industry-wide. Below, we’ll consider the issues facing observability and introduce OpenTelemetry, a CNCF project quickly gaining traction to unify telemetry data from various sources.

    Open Standards For The Win

    Although proprietary monitoring agents have a lot of intelligence, they inherently use tight coupling between telemetry generation and the backend storage analytics, said Horovits. This can cause vendor lock-in and lead to formatting variations in telemetry data between toolsets. “The open standards have the power to converge our industry to prevent vendor lock-in, even the playing field, and allow us to get any telemetry for any source,” said Horovits.

    Without an open standard, supporting the evolving suite of technologies at the heart of modern polyglot software stacks is challenging. Organizations constantly add to their tech stack, from SQL to NoSQL, Kafka, API gateways and serverless cloud services, and these modules produce varying data formatted in unique styles. What’s more, companies are typically using more than one cloud vendor, further necessitating the need for a decoupled method to represent multi-cloud metrics. “As soon as your IT stack becomes this complex, there is a need to aggregate all telemetry into one unified observability standard,” explained Horovits.

    Introducing OpenTelemetry

    OpenTelemetry is described as a “high-quality, ubiquitous and portable telemetry to enable effective observability.” Using OpenTelemetry (sometimes abbreviated as OTel), engineers can generate, collect and export metrics, logs and traces. OpenTelemetry works with many popular libraries and is completely vendor-neutral and supported by a rising number of tools in the space. OpenTelemetry was accepted to CNCF on May 17, 2019, and is an incubating project.

    The OpenTelemetry registry boasts hundreds of libraries and plugins to export and transform telemetry data from many different environments, from legacy to contemporary sources. By decoupling the data collection and export process, OpenTelemetry is “one agent to rule them all,” Horovits explained. Of course, he added, vendors who adopt OpenTelemetry may still provide their own logic behind it.

    The Benefits of More Open Telemetry Data

    At the time of writing, live public repository data shows OpenTelemetry is the second-most active CNCF project, behind Kubernetes. So, why is the observability community so excited about OpenTelemetry? Well, for one, the interoperability it affords could equate to efficiency and time savings. “OpenTelemetry gives you the freedom to integrate throughout the stack,” stated Horovits. Part of this is because its open source roots make it a force multiplier. “You can never achieve such vast coverage with closed source as you would with a wide-open community,” Horovits said.

    Arguably, an open telemetry toolset has also become necessary due to the sheer complexity of cloud-native architectures. Most organizations are shifting to a distributed microservices architecture supported by a dynamic, elastic container-based infrastructure. In this world, a single service might span multiple clusters. Therefore, engineers might need to consider clusters, pods, nodes and namespaces when filtering for data.

    In this complex reality, there are many dimensions, multiplying what needs to be monitored. And in general, traditional monitoring systems become ineffective at this level. Tool sprawl and lack of consolidation are problems that arise when trying to cater to many environments simultaneously. Thus, a unified, decoupled observability platform makes a lot of sense.

    Future of OpenTelemetry

    We’ve seen vast adoption from cloud vendors and solution providers around OpenTelemetry. And, the project has been moving swiftly—OpenTelemetry has had a distributed tracing specification released for about a year. The project almost has a release candidate for metrics and one for logs is in the works, said Horovits.

    Beyond that, one feature that Horovits is excited about is continuous profiling, which could help users continuously analyze a function and track usage trends across instances. Another exciting potential development is eBPF. Since it’s language-independent and runs at the kernel level, eBPF could extract observability signals without requiring application-specific hooks or integrations. This could increase support for more environments.

    For more information, you can visit the OpenTelemetry site to peruse the documentation or read up on the feature roadmap for OpenTelemetry. Horovits also gave this helpful presentation on OpenTelemetry at KubeCon 2022.

  • Cloud-Native: It’s One Thing

    Cloud-Native: It’s One Thing

    This Wednesday, August 10, 2022, starting at 9:00 a.m. ET, Techstrong Group is hosting an awesome virtual conference: CloudNativeDay. I am sure most of you already know a lot about the many cloud-native solutions out there; are familiar with the great work the Cloud Native Computing Foundation (CNCF) does and that you have Kubernetes clusters running in your environments. But for me, at first, the interest and enthusiasm people showed for anything and everything cloud-native defied my understanding.

    We do about one virtual conference a month here at Techstrong Group. That may sound like a lot, but when you divide it up among topics in DevOps, cybersecurity, cloud-native and digital transformation and then further segment who in an organization is targeted by those events, it doesn’t give us a lot of room to dive as deeply into some topics as we would like. We pick the topics of our virtual events carefully and we depend on feedback from you, our audience, to guide us.

    Tim Hockin, Principal Software Engineer: Kubernetes, GKE, Anthos at Google

    This is the fourth year we are producing a cloud-native virtual event, so the topic has obviously resonated with many of you. But it wasn’t until this past year that I had the “Eureka!” moment and realized why.  You see, to discover the reason why cloud-native was so popular, I looked in the usual spots: Software, applications and solutions. I felt confident that Kubernetes’ popularity was one of the reasons cloud-native was so popular. Over the last 24 months, observability burst on the scene. Maybe that was driving the continued popularity of cloud-native. Heck, the CNCF has over 40 projects; perhaps the sheer quantity is what makes cloud-native so popular. But none of that felt exactly right—yes, all of these things drive cloud-native’s popularity. But they aren’t the key driver.

    Once again this year, I had the pleasure of recording a fireside chat with Tim Hockin. If you don’t know him, Tim is a principal software engineer at Google and an original member of the team that brought Kubernetes out from Borg (if you don’t know Borg, then you need to learn some cloud-native history). This is the fourth year we have Tim keynoting this virtual conference, and chatting with him is always a highlight of the event for me.

    Tim said something in our conversation this year that finally made it click for me. Suddenly, I had found it. The “one thing”, in the immortal words of Curly from the movie City Slickers.

     

    You see, the secret to cloud-native isn’t how great and powerful Kubernetes is; it’s not what observability or service mesh or any of those great cloud-native or CNCF-developed software tools can do for you. Cloud-native is a state of mind. As much as it is about software, it is also a state of mind. Cloud-native is about building powerful, applications at scale using the entire ecosystem that was created as a result of that state of mind.

    Sound a little too zen for you? Well, I hope to change that thinking. You do not want to miss my conversation with Tim Hockin that kicks off our show this year. We’ll discuss the belief and help you understand why cloud-native is a state of mind. And, while you’re listening, Tim and I also talk about what is new in fleet and multi-cluster Kubernetes.

    The entire virtual event program is chock full of great sessions. Head over to the event site and see for yourself. Panel discussions, speakers, demos. Great exhibitor booths you can visit and ask questions. Games, prizes, a DJ, a mixologist and more. We can’t wait to see you there!

    And if you remember anything from the event, let it be this one thing: Cloud-native is a state of mind.

  • Learn Something New Every (Cloud-Native) Day

    Learn Something New Every (Cloud-Native) Day

    Being the new guy at Techstrong Group, the first couple of weeks was spent trying to find things. Like the bathroom when I’m at HQ in Boca Raton or the login to the SurveyMonkey account so I could check on the status of our various in-flight research projects. 

    I also realize I have to learn the vernacular, especially around DevOps and this cloud-native stuff. To be clear, I’ve been researching DevOps and how to integrate security into the process (yes, DevSecOps) for a while. But this cloud-native terminology is something I need to learn more about.

    You’re probably a bit confused about why I’m perplexed by the term ‘cloud-native.’ My work in cloud security for the past 10 years gave me some opinions about vernacular. When I say “cloud-native,” I’m typically referring to the native services you use in your cloud provider. Like AWS Fargate or Azure Container Registry. Those services are “native to the cloud (provider).” 

    But I’m realizing that native cloud services are only a portion of the larger cloud-native revolution. Like DevOps, cloud-native is a state of mind, not a service you buy from a cloud provider. It’s about openness. It’s about portability. It’s about agility. It’s also humbling as I realize I have a lot more to learn.

    That’s why I’m looking forward to our CloudNativeDay event on August 10, 2022. I get an opportunity to dig in and learn about some topics germane to my new coverage areas. For instance, the session on cloud-native challenges in traditional enterprises caught my eye. I’ve been involved in born-in-the-cloud companies, so we didn’t have any installed base to consider. But how would you embrace new and disruptive technology when your company has extensive legacy infrastructure? I can’t wait to hear about those more traditional enterprises’ opinions and experiences.

    Observability is also high on my list of things to learn more about. I’ve been working with customers on advanced security monitoring strategies for decades, but looking at the application layer and associated infrastructure is a different animal. I suspect there will be some technical similarities (collect data, analyze data, alert on data, etc.), but the use cases are fundamentally different. 

    I’m also excited for the last keynote of the day. However, I’m a little biased since it involves an excellent conversation Mitch Ashley and I had with Donald Lutz of Taos (an IBM company), where we dug into some of the enterprise migration challenges he’s seen in the field. 

    They say that when you’re young, you know everything. But as you get older, you realize you don’t know anything. I’m definitely in the latter camp, which is why I’m not going to squander a great opportunity to learn—and neither should you. Register and check out the event. I’ll see you there.

  • Deloitte Aligns with Dynatrace for Observability

    Deloitte Aligns with Dynatrace for Observability

    Deloitte has expanded its cloud observability practice to include the Dynatrace Software Intelligence platform.

    Jay McDonald, managing director and co-chair for modern delivery at Deloitte, said the IT services provider will now provide traditional DevOps and observability consulting expertise along with a set of instances of the Dynatrace Software Intelligence Platform that it will manage on behalf of organizations.

    Deloitte opted for the Dynatrace platform because it employs a single agent for collecting metrics, logs and traces that are then analyzed by its Davis artificial intelligence (AI) engine to enable observability, said McDonald.

    While there is a growing appreciation for observability to enable DevOps teams to identify the root causes of IT issues before they become major problems, deploying an observability platform requires a lot of specialized expertise. Many organizations are opting to rely on third-party organizations such as Deloitte to manage that task within the context of a larger DevOps workflow, said McDonald.

    Managing those workflows has become especially challenging for many organizations to deploy and maintain as they move to build and deploy microservices-based applications, he added.

    Michael Allen, vice president of worldwide partners for Dynatrace, said there are now already more than 4,000 organizations using the Dynatrace Software Intelligence Platform, a number that will continue to increase as observability platforms are consumed as a service.

    It’s not clear at what rate DevOps teams are embracing observability. The concept has always been a core DevOps principle, but, for the most part, more mature DevOps teams have only been able to achieve what amounts to continuous monitoring of pre-defined metrics. Observability platforms promise to surface anomalies indicative of IT issues before they escalate; DevOps teams can then launch queries to better determine their root cause and potential severity.

    One of the benefits of the Davis AI engine is that machine learning algorithms surface the root cause of many of those issues without requiring DevOps teams to know exactly how to construct the right query, noted Allen.

    However, the biggest challenge many DevOps teams encounter is they lack the skills required to deploy and manage observability platforms. In fact, many of them are now relying on third parties to manage a wide range of DevOps platforms so they can concentrate their efforts on actual workflows. Hiring and retaining DevOps professionals also has become more challenging as the total number of applications being built and deployed expands beyond DevOps teams’ abilities to effectively manage.

    One way or another, the days when IT teams determined the root cause of an IT issue by process of elimination are finally coming to an end. DevOps teams are gaining more visibility into their application environments in ways that surface more meaningful actionable intelligence, regardless of who sets and manages the observability platform. The opportunity to employ that intelligence to improve overall application resiliency and drive a wide range of digital business transformation initiatives is well within DevOps teams’ reach.

  • Connections Matter: Gaining Visibility Into APIs

    Connections Matter: Gaining Visibility Into APIs

    Digital transformation is at the center of modern enterprise. Yet, as cloud initiatives take shape and the complexity of connecting systems and applications multiplies, the need to tie everything together and automate processes grows. APIs, otherwise known as application programming interfaces, fuel this automation. These small pieces of code—which connect and interconnect clouds, containers, microservices and applications—help organizations compress what used to be years of project development into months or weeks.

    As businesses connect functionality—often across companies and service providers—APIs lead to an enhanced user experience. Simply put, they make it possible for much of the magic of the internet to take place, including seamless authentication, data sharing, payments and collaboration.

    Today, it’s next to impossible to find an enterprise application that doesn’t rely on APIs. Yet, despite all the gains, APIs can cause plenty of pain. The slow performance or complete failure of an API can lead to an array of problems. These include customers who can’t complete transactions, employees who can’t collaborate through critical apps, and incomplete or incorrect supply chain data.

    Today, a typical organization relies on more than 1,200 cloud-based applications and hundreds or even thousands of APIs. For many businesses, 50% to 90% of revenues are linked to APIs. If a system underperforms or breaks, there’s often no way to identify the root cause and fix it promptly. Reputational damage and lost sales often ensue.

    When Good APIs Go Bad

    It’s important to recognize that the monolithic applications of the past are a vanishing breed. Today’s highly-distributed software and services environment taps a universe of more than 24,000 APIs. It isn’t unusual for a transaction to pass through several cloud containers and across numerous companies en route to the desired outcome. Along the way, these APIs encounter different hardware, code and conditions.

    When APIs work well, a process or transaction typically finalizes within milliseconds—and the digital experience usually excels. However, if a page loads slowly, a promotional code doesn’t work or payment doesn’t go through, the odds are high that somewhere along the line something went astray with an API.

    For many organizations, there’s no simple and fast way to identify the source of the problem, understand the current impact, and take steps to fix the failure post haste. In fact, it isn’t unusual for several thousand or more failed transactions to pile up before the issue hits IT’s radar and anyone begins to notice a problem. By then, shoppers have likely abandoned shopping carts and sales have evaporated.

    Adding to the challenge is the fact that APIs can fail in several ways. Not surprisingly, the number one problem is availability. If the API doesn’t function, the interaction or transaction cannot take place. In addition, an API might perform slowly or erratically, thus delivering highly inconsistent and frustrating experiences for users along with a problem that is difficult to diagnose and fix. Yet another problem is misconfigured or poorly functioning APIs that return incomplete data or the wrong data.

    The real-world fallout can be significant. In practical terms, a customer shopping at a clothing retailer’s website may find it impossible to add socks to a shoe order or use one or more payment services. It may be impossible to cash in points from a loyalty program or view delivery information. Broken APIs can disrupt or break shopping carts, customer accounts, supply chain data, collaboration links, security tools, and much more.

    Of course, when things go haywire, finger-pointing and conflicts with vendors, business partners and customers often ensue.

    Deep Visibility

    In today’s API-centric world, detecting problems early and often is critical. What makes the task so difficult is that API transactions do not unfold in a linear way. They bounce across servers and companies—often a couple dozen of them—scattered all over the world. Further complicating things: This routing can change at any given instant.

    Identifying specific issues that trigger a problem—traffic bottlenecks, bandwidth issues, code problems and more—requires the right tools. It’s vital to understand how an API-based transaction behaves across a dozen or even two dozen hops and touchpoints that extend through clouds, containers and microservices.

    Passive visibility tools, while useful for getting closer to the end-user, do not offer a comprehensive and granular view of events and processes. Today, businesses require broader and deeper observability capabilities that extend across an ecosystem of APIs and into the entire IT landscape—from network to device. It’s vital to understand how customers use websites, apps and other services.

    Making Integrations Matter

    As organizations delve deeper into clouds and digital interconnectivity, a more nuanced approach that incorporates both passive and active observability is in order. This approach makes it possible to emulate a user journey and quickly identify problem points. A business can detect problems as they pop up—after only a handful of failed transactions—rather than thousands or even tens of thousands of transactions later.

    With a best-practice observability and remediation framework in place, an enterprise can establish baselines and triggers for alerts and notifications. It can assemble a complete picture within a single pane of glass and troubleshoot across the delivery chain. Whether the API involves a collaborative task within the organization or a financial transaction that stretches across several companies, it’s possible to view the entire transaction—and experience.

    With both passive and active monitoring in place, a business can evolve beyond constant fire-fighting mode. Armed with the information and insight required to make vendors more accountable, businesses ensure they are getting the level of service (SLA) they are paying for. Make no mistake, as digital transformation initiatives accelerate and cloud complexity grows, it’s critical to ensure that APIs, microservices and other digital components are up to speed.