devopsy.kinsta.cloud

Tag: alerting

  • The Right Tool for the Job: Alerting Edition

    The Right Tool for the Job: Alerting Edition

    In high-tech, we have a terrible predilection for finding a good tool and abusing it until we hate it. I’ve written several “Use the right tool for the job” articles and posts over the years, most notably an examination of trying to make one programming language (or another) into an all-purpose solution. We’re thankfully past that … for now. But we do the same thing across all of IT in general and DevOps in particular. Consider what we did to Jenkins when DevOps was coming of age. It went from a great innovative tool to a bloated monster we tried to use to automate just about everything short of making coffee in between integration runs. Fact is, Jenkins is still a good tool in the CI space; everyone started hating it because it was over-used. It’s a terrible tool when used for things other than what it’s good at. Imagine that. We can make the same statement about, well, everything. “A car is a terrible tool when used as a life preserver,” is a good example of some of the absurdity that makes us hate good tech.

    The 2023 edition of this phenomenon came up on a for-fun video call with some friends this weekend, and talking about it made me realize how bad it has become. Thus, I give you the point of this post:

    Slack is not your documentation tool. It is not anything but an efficient alerting tool and maybe a stand-in communications tool for short-term convos.

    I see it all over the place. “Oh, we discussed that on Slack.” That’s great, where did you document it? Because honestly, some of the companies I deal with have busy enough Slack channels that if the conversation happened more than an hour ago, it will never be seen again.

    Slack, and a few similar tools, are astounding alerting channels, offering an opportunity to notify when events occur, and we really should continue to reap the ease-of-use and massive alerting benefits that these tools offer. But don’t confuse that with long-term anything. Because it is not.

    Alerting to Slack needs to be one piece of a larger puzzle. Making sure that whatever was alerted on was also logged somewhere (either by the alert mechanism or by whatever tool spawned the alert), and for a whole array of issues that are alerted on and many of the conversations that happen in Slack, long-term documentation needs to be created.

    So this is a lot of words to remind you of what you already know: Slack can enable quick communications, alerting and innovative solutions by having broader discussions without cramming everyone into a conference room. But it is not a documentation tool in any way, and it needs to be part of an integrated solution that uses the right tool for the job. Security alerts, for example, better be ending up in the SIEM (they are in every organization I’m aware of; just using that as a glaring example).

    And keep rocking it. Just make sure while you’re running at 500 miles per hour to keep up with Agile and DevOps speeds, you are capturing the stuff that will be important in six months or a year somewhere that people can actually find it. And use each tool to do what it’s best at. Convenience is almost never the best selection criterion for a tool to solve a given problem.

  • Blameless Integrates Incident Management Platform With Opsgenie

    Blameless Integrates Incident Management Platform With Opsgenie

    Blameless this week announced it has integrated its namesake incident management solution with the Opsgenie alerting platform from Atlassian.

    Aaron Lober, director of product marketing for Blameless, said the Blameless platform is now integrated with both Opsgenie and PagerDuty, the two most widely used platforms for managing alerts.

    Blameless provides an incident management platform that normalizes data collected from a wide range of DevOps tools and platforms to determine how best to respond to different incident classes. Armed with those insights, it then becomes possible to quickly assemble the best team to respond to an event, said Lober.

    That capability reduces stress because Blameless also surfaces recommendations for resolving incidents based on the data it has collected, he added.

    The Blameless platform is tailored to organizations that have hired site reliability engineers (SREs) to manage IT operations using DevOps best practices, noted Lober. The overarching goal is to eliminate rote processes that, over time, conspire to burn out incident management teams that often experience high turnover rates, he added.

    Opsgenie has been connected to the Blameless platform via the service catalog and alerting functions. Blameless users responding to or managing an incident can escalate issues to any teams defined within Opsgenie. The service catalog and alerting data generated throughout the life cycle of an incident is automatically captured and stored on the Blameless Incident Timeline. That capability makes it possible to identify appropriate service owners and the escalation protocol that was followed by that team. Blameless can also display the actual user name of each responder that has been notified.

    When accessing Blameless via Slack or Microsoft Teams, Opsgenie can automate both when an incident channel is created and pre-select relevant services that should be made available. Alerts can also be manually triggered using commands from the Blameless chatbot for those who prefer to trigger alerts from within Slack or Teams themselves.

    As IT environments become increasingly more complex, more organizations are making a concerted effort to apply DevOps best  practices to incident management. The goal is to reduce downtime and increase overall resiliency at a time when organizations are more dependent on software than ever. As part of that effort, IT teams should identify the repetitive processes that span multiple incidents and then move to automate as many as possible.

    In the longer term, it’s not clear what impact artificial intelligence (AI) will have on incident management. At some point, though, IT teams should expect their efforts to be augmented by machine learning algorithms capable of identifying issues before they balloon into a major crisis. However, there will always be some human intervention required to manage incidents for the foreseeable future, noted Lober.

    In the meantime, most IT teams should continue to expect the unexpected at a time when IT environments have never been more dynamic and change multiple times a day.

  • SRE’s Guide to Pragmatic Incident Response

    SRE’s Guide to Pragmatic Incident Response

    In my past experience as an SRE, I learned some valuable lessons about how to respond to and learn from incidents. If you want the TL;DR, I’ll summarize them here:

    • Declare and run retros for the small incidents. It’s less stressful, and action items become much more actionable.
    • Decrease the time it takes to analyze an incident. You’ll remember more, and will learn more from the incident.
    • Alert on pain felt by people – not machines. The only reason we declare incidents at all is because of the people on the other side of those machines.

    Now, let’s dive into each of these lessons a little deeper, and explore how they can help you build a better system for pragmatic incident response.

    1. Focus on the Small Incidents First

    The bad habit of ignoring small issues often leads to bigger issues. You should run retrospectives for small incidents (slowdowns, minor bugs, etc.) because these often have the most actionable takeaways—instead of shooting the moon and creating “rearchitect async pipeline” as a Jira ticket that never happens. Focusing on low-stakes incidents and retrospectives are a great introduction to behavior change across your organization.

    Let’s look at an obvious example of how a focus on small incidents can have an outsized impact. Case in point—my apartment. I live in an old candy factory retrofitted into apartment units. We have an elevator (thank God), but some of the buttons don’t light up when you press them. The LED display doesn’t match up with the numbers on the floor buttons. Yes, the elevator goes up and down, but overall, you can tell that things are wrong with it.

    One day, I came home and noticed an “Out of Order” notice on the elevator door. Was I surprised? Not at all. Of course an elevator with mislabeled buttons and broken LEDs would stop working!

    This isn’t dissimilar from an important software lesson—ignoring small issues often leads to much bigger issues. What starts as an inconvenience can lead to a complete breakdown. This is why you should focus on the small incidents first. I hear a lot of companies say, “We need to fix incident response! Every time we have an incident, it’s just chaos!”

    They are talking about unexpected downtime during a high severity level incident and they want to improve it.

    I say that’s the wrong type of incident to use to fix your incident response system and process. Focus on the small incidents first. Running retros for small incidents can help you build strong incident response models, because they have the most actionable takeaways and they are the best way to change behavior through repetition and practice.

    If you have a high-stakes, 12-hour incident and you run a one-hour retrospective you’re not going to get the results you want. You need to start small. Run retros for bugs that were introduced; maybe a bad data migration that didn’t really impact anything but took up a couple of hours of your day.

    Heidi Waterhouse captured this idea really well in this piece on reliability. Every airplane you’ve ever flown on has many tiny problems, Waterhouse said, “… like a sticky luggage latch or a broken seat or a frayed seatbelt. None of these problems alone are cause to ground the plane. However, if enough small problems compound, the plane may no longer meet the requirements for passenger airworthiness and the airline will ground it. A plane with many malfunctioning call buttons may also be poorly maintained in other ways, like faulty checking for turbine blade microfractures or landing gear behavior.”

    I couldn’t agree more. Extrapolating to software: The small things are typically indicators of bigger issues and could cause catastrophes down the road.

    2. Track Mean Time to Retro (MTTR)

    It’s important to think about what you measure in your organization. You should be measuring how you’re improving, and the most important metric here is mean time to retro (MTTR). Everyone should be tracking MTTR. It’s a great statistic to improve incident response in your team because it helps you understand the delay between incidents and retrospectives.

    The easiest way to have a bad incident retro is to wait two weeks. It’s better to get into a room quickly and hash out what happened than wait a long time until you’ve got everything perfectly prepared.

    Tracking MTTR can help you hold prompt and consistent retrospectives after incidents. Set a timer and make an SLO or SLA for yourself that says, “This is how long we take for retros.”

    Retro time will vary depending on the severity of the incident itself. If it’s a SEV1, clear everyone’s schedules, because you need to have retro within 24 hours of the incident. For SEV3 incidents, you have much more leniency.

    I also like tracking the ratio of retros to declared incidents. This is a metric that should go up; you should see your ratio of retrospectives to incidents increasing. You can break down that number by severity, as well. If you have a low retro ratio for a SEV1 versus a SEV3 incident, that might be okay at first (remember, start small) but you want those metrics to eventually become equal.

    3. Alert on Degraded Experience with the Service, and Not Much Else

    The severity of incidents is directly linked to customer pain. We would not declare SEV1s if there weren’t a lot of people feeling a lot of pain.

    Alerting on computer vitals is an easy way to create alert fatigue and burnout. As your company starts to scale, you are going to use more CPU; you’re going to use more memory. Tying alerts to computer vitals is not a good sign.

    If I run, my heart will beat faster—it’s just doing its job. Paging people at 2:00 a.m. because disc capacity is at 80% even though you won’t run out of space until next month is a good way to lose great teammates. I have worked with people who left their previous companies strictly because they got paged too many times for stuff that didn’t matter.

    This is why you need to alert on a degraded experience with the service and not much else. A CPU burning hot at 90% is not necessarily a bad thing, but you need context to decide. Create SLOs that are tied to customer experience and alert on those. People experiencing problems with the service is the only thing you should alert on—for the most part.

    One of the best ways I’ve seen to think about this came from Soundcloud developers, who explained that you should alert on symptoms, not causes. My fast heartbeat is not necessarily a problem. But if my elevated heart rate leads to lightheadedness and I fall—that’s a problem. So, I need to be able to alert on something like that. Paging people at 2:00 a.m. because disc capacity is at 80% and you won’t run out until next month is not good. But paging people because you know that disc capacity problems cascade into other, systemic problems is. You can apply the same thought to other potential causes of an outage. Paging alerts that wake you up in the night should only be based on symptoms.

  • CircleCI Bets on Webhooks to Streamline DevOps Integrations

    CircleCI Bets on Webhooks to Streamline DevOps Integrations

    CircleCI today announced CircleCI webhooks, a feature that enables software engineering teams to build integrations that react to CircleCI job and workflow status notifications in near-real-time.

    Tom Trahan, vice president of business development at CircleCI, said this latest iteration of the company’s implementation of webhooks, which send messages any time an event occurs, will become the foundation upon which CircleCI will expand the ecosystem surrounding its continuous integration/continuous delivery (CI/CD) platform.

    For example, the CircleCI platform can now employ webhooks to integrate with a CI visibility tool from Datadog that can now be used to surface CicleCI metrics. Sumo Logic is similarly employing webhooks to enable its Continuous Intelligence observability platform to track events on the CircleCI platform. Those capabilities are now available via the software-as-a-service (SaaS) edition of the CircleCI platform.

    Trahan said the primary goal is to reduce the amount of friction within DevOps workflows that span multiple platforms by increasing the level of orchestration DevOps teams can take advantage of without having to create and maintain integrations using low-level application programming interfaces (APIs).

    Webhooks make it possible to streamline developer workflows while also increasing overall engineering velocity, he added. In addition, it should also become simpler for DevOps teams to triangulate events in a way that results in incidents being resolved faster, noted Trahan.

    CircleCI is relying on webhooks to strengthen its case for a best-of-breed approach to DevOps that enables organizations to mix and match tools as they best see fit rather than getting locked into a single integrated DevOps platform. The best-of-breed approach often makes it simpler to take advantage of new tools as they become available. The downside is the level of integration that DevOps teams need to maintain after they integrate all the tools that make up a DevOps workflow.

    It’s not clear what impact relying on webhooks to share messages in real-time about events as they occur across multiple platforms will have on integrations. However, any effort to streamline DevOps environments without requiring DevOps teams to abandon one tool for another is likely to be well-received. After all, every minute spent managing a DevOps platform means much less time that can be applied to building and deploying applications.

    It will be up to each DevOps team to decide to what degree a best-of-breed approach to DevOps suits their organization. As more organizations realize how dependent they have become on software to, for example, drive digital business transformation initiatives, many IT leaders are making a case to standardize DevOps platforms across their organization. Achieving that goal, however, may be difficult given all the various departments within an enterprise that have already adopted a wide range of tools and platforms.

    In the meantime, the focus on developer and engineering productivity will only increase as organizations look for ways to further accelerate application development and deployment at a time when software updates can now have a profound impact on profits and revenue. In fact, a delay to one software project can have a cascading impact on a wide range of initiatives that can’t go forward because of their dependency on a delayed project. Given the impact all those delays might have on revenue, the tolerance organizations have for delays to software development projects is understandably lessening by the day.

  • xMatters Streamlines Incident Management

    xMatters Streamlines Incident Management

    xMatters today announced it has expanded the capabilities of its incident management platform to better address the requirements of more complex IT environments.

    New capabilities include an Incident Console to better visualize and triage incidents, automate responses, improve collaboration and access to richer analytics to optimize response times. Incident metrics are now also automatically collected from integrated systems and surfaced along with the notification to improve resolution times.

    IT teams can also use a Flow Designer tool to automatically keep related systems current with one another, including issue tracking systems, service desks and status pages.

    xMatters CEO Troy McAlpin said the updates are part of an effort to unify IT incident management at a time when the adoption of new applications and systems to enable organizations to have employees working from home has increased.

    A survey of 300 IT leaders and business professionals published today by xMatters revealed nearly three-quarters of respondents (72.3%) now spend at least half of their time resolving incidents, with more than a quarter (27.3%) of those respondents saying they their team spends at least 80% of that time resolving incidents. A total of 43% of respondents said their organization still relies on less sophisticated processes to manage IT incidents, such as alerts, email, paging, conference bridges and physical outreach.

    IT incidents are also starting to have a bigger impact on the business, noted McAlpin. The survey finds 20% of companies with 1,001-5,000 employees are budgeting more than $10 million on digital transformation initiatives. Each IT incident involving those applications now typically involves not just employees but also end customers, he noted.

    As IT becomes more complex, he said, the need to modernize IT environments has become crucial. One of the many reasons IT organizations find it difficult to automate IT management is there are still many unique platforms running isolated applications. IT teams would be well-advised to ask themselves if these unique instances of a platform are truly warranted. If not, that application workload should be retired or shifted to a platform that is more easily automated.

    Longer-term, McAlpin said artificial intelligence (AI) will increasingly be employed across IT operations, otherwise known as AIOps. Those advances won’t replace the need for humans to manage IT, but it will allow IT teams to better automate responses to routine incidents that today require too much time.

    Even as IT becomes more complex, the simple truth is most organizations are not looking to expand the overall size of the IT department. In fact, in the wake of the economic downturn brought on by the COVID-19 pandemic, there are many organizations that need to reduce the size of their IT organizations. Regardless of motivation, there are a host of mind-numbing tasks that IT teams are asked to perform every day that, in keeping with best DevOps practices, should be ruthlessly automated.

  • LogicMonitor Embraces AIOps

    LogicMonitor Embraces AIOps

    LogicMonitor today announced the general availability of LM Intelligence, an extension of its monitoring platform that leverages machine learning algorithms to identify potential issues that are indicative of a major impending outage.

    Sarah Terry, manager of product management for LogicMonitor, said rather than acquiring a separate AIOps platform, it makes more sense to employ machine learning algorithms within the context of a tool that DevOps teams already are using to monitor their IT environment. Monitoring platforms collect the massive amount of data required to train those algorithms, but a separate AIOps platform has to find a way to collect that data, noted Terry.

    The LM Intelligence module created by LogicMonitor provides a root cause analysis (RCA) capability that automatically discovers infrastructure topology in both cloud and on-premises IT environments. Once an issue is discovered, LM Intelligence will aggregate all the alerts being generated into a single notification that pertains to the root cause of any potential issue. That capability will substantially reduce the amount of alert fatigue that today results in IT teams often disregarding what could be a critical alert that would have averted a system failure, said Terry.

    DevOps teams can also employ LM Intelligence to define expected performance ranges for IT resources based on historical trends. Any anomaly outside those ranges will generate a notification.

    As IT environments become more complex, thanks in part to the rise of multi-cloud computing and microservices-based applications based on containers, it won’t be feasible for the average IT team to identify all the relevant events that might be impacting that IT environment without help. There are simply too many parameters that need to be tracked. Tools such as LM Intelligence don’t replace the need for IT operations expertise as much as they augment the capabilities and expertise of the teams trying to manage complex IT environments, where there are large numbers of dependencies between systems. In effect, Terry said, tools such as LM Intelligence will enable DevOps teams to identify the signal in all the noise being generated within an IT environment.

    As IT management tools continue to evolve, DevOps teams should expect to see reliance on machine learning algorithms to become more pervasive. Rather than being a standalone platform, AIOps is quickly morphing into a feature that will become a capability embedded across a wide range of tools.

    Of course, each DevOps team will need to decide to what degree they want to trust any form of AI. However, the longer algorithms collect and analyze data, the more accurate they tend to become. That doesn’t mean DevOps teams should automate processes based on the recommendations of an AI engine; there are likely to be parameters that an AI engine may not have accounted for that could have an adverse impact. However, having the equivalent of a digital assistant that provides timely advice may soon prove indispensable.

    — Mike Vizard

  • Trend Micro Partners With Snyk to Advance DevSecOps

    Trend Micro Partners With Snyk to Advance DevSecOps

    Trend Micro today announced an alliance with Snyk through which alerts about vulnerabilities in open source code will be passed on to the tools Trend Micro makes available to apply virtual patches to both monolithic and microservices-based applications.

    Snyk provides a tool that identifies and fixes vulnerabilities and license violations in open source dependencies and container images. Trend Micro COO Kevin Simzer said his company leverages the alerts generated by Snyk to inform developers and cybersecurity professionals where virtual patches need to be applied.

    Virtual patches are a capability Trend Micro developed to enable IT organizations to address a vulnerability by applying a security policy that limits access to a specific piece of code until a developer can update a monolithic application or replace the containers housing vulnerable code within a set of microservices.

    By working more closely with Snyk, Trend Micro is trying to increase the appeal of its approach to virtual patching among DevSecOps teams, said Simzer. It’s not possible for most developers to address every vulnerability that manifests itself in a timely manner. Virtual patching allows DevSecOps teams to establish which vulnerabilities need to be addressed immediately while making use of virtual patches to protect applications from less severe vulnerabilities until they can be addressed during a DevOps cycle. To advance those capabilities, Simzer noted Trend Micro has already integrated its security offerings with continuous integration/continuous deployment (CI/CD) platforms such as Jenkins.

    Simzer said it will take a while for most cybersecurity professionals to trust developers to address cybersecurity issues during the application development process. However, virtual patches provide a mechanism through which both cultures can collaborate more closely without compromising the rate applications are being built and deployed. The primary reason developers tend not to pay as much attention to cybersecurity issues as everyone would like is because of the deadline pressure most of them are operating under. However, as responsibility for application cybersecurity continues to shift left, more developers are trying to address these issues earlier in the application development process.

    More than 16,000 enterprise IT organizations are already using Trend Micro security offerings to secure both monolithic applications and cloud-native applications based on containers, said Simzer, adding that footprint puts Trend Micro in a position to help organizations embrace best DevSecOps practices.

    Of course, Trend Micro is not the only cybersecurity vendor with similar ambitions. The challenge many organizations will face is defining what processes are best handled by developers invoking application programming interfaces (APIs) versus cybersecurity professionals who are more comfortable employing graphical user interfaces (GUI). There may come a day when more cybersecurity professionals are able to write code, but for now the two camps could not be further apart in terms of the tools they prefer to use. For IT security vendors such as Trend Micro, that means for the foreseeable future, they will continue to need to build GUIs alongside APIs.

    — Mike Vizard

  • Alerting Consolidation: What’s the DevOps Impact?

    Alerting Consolidation: What’s the DevOps Impact?

    Over the last 12 to 18 months, the monitoring and alerting piece of the DevOps toolchain has been going through some serious consolidation. Companies are bracing to compete with the world or getting picked up by larger DevOps players, making their product more appealing through integrations.

    In general, if such changes don’t impact your toolchain, I say ignore them. The exception is when it appears the entire submarket might need to get aligned to survive, because that means at some point your toolchain likely will be affected.

    So, to recap the source of this blog, in the last short while OpsGenie has become a part of Atlassian, VictorOps has become a part of Splunk and PagerDuty has filed for an IPO. It is likely the rest of the market will have to match these three or admit to eroding market shares. In a competitive market where customers are using a fraction of the available functionality of a given product, deeper integrations (like come from ownership) or bold road maps (like come from IPOs) can have an impact on what customers choose to use.

    Alerting—and monitoring in general—is still relatively new to its current incarnation, and unlike some markets there are still very real differentiators in it. Most alerting engines have a core of functionality built in, while a lot more functionality can be had through integrations with other products. Some focus on the network to garner the bulk of their alert conditions, while others focus on application performance and still more watch the infrastructure closely. While all do a bit of each of these things, they still definitely have a focus. That will change over time as the market either settles down or fragments. I’m hoping it will settle down; we have enough sub-markets in DevOps already.

    All I’m suggesting is that you stay in contact with your vendor of choice and watch the overall market. Some of the changes might create a situation in which an alternative is appealing, and that is worth looking into, but other changes might negatively impact your chosen solution, which is absolutely worth noting.

    If your product of choice is clipping along in a way that is acceptable/useful to you, by all means stay with it. There’s no sense changing a product simply because of possibilities. But being aware can help you navigate the market changes that are well underway and very well might hit you right in the stack.

    — Don Macvittie

  • Atlassian Integrates DevOps Portfolio

    Atlassian Integrates DevOps Portfolio

    At its Atlassian Summit 2019 conference, Atlassian announced updates to all the DevOps tools that make up Atlassian Data Center as well as upgrades to its Opsgenie incident management software to provide deeper integration with the rest of the company’s portfolio.

    At the same time, Atlassian revealed it has rebranded the AgileCraft business planning application it acquired earlier this year to Jira Agile.

    New capabilities added to Atlassian Data Center include the ability to set up smart mirror farms for Bitbucket Data Center; content delivery network (CDN) support for Jira Software, Confluence, Bitbucket and Jira Service Desk Data Center; rate limiting for Bitbucket, Jira Software and Jira Service Desk Data Center; issue archiving for Jira Software and Jira Service Desk Data Center; templates for deploying on Microsoft Azure and Amazon Web Services public cloud; and a mobile application for Jira Server and Data Center.

    Atlassian Data Center now incorporates Jira Software 8.0, Confluence 7.0, Bitbucket 6.0, Jira Service Desk 4.0 and Portfolio 3.0. The latest version of Jira Software 8.0 now executes advanced searches using Jira Query Language (JQL) 31% faster on average, while agile boards load two times faster and re-indexing times are now 71% faster. The latest version of Confluence adds support for multimedia files and additional analytics tools to the content management platform.

    The Atlassian Cloud service has also been expanded to increase the user limit to 5,000, with an early access program signaling an intention to bring that up to 10,000 users soon.

    Meanwhile, specific capabilities being added to OpsGenie include support for incident timelines to track response activities and key events; postmortems to help discover root causes and track remediation activities, a user interface that aligns with the rest of the Atlassian portfolio that fosters bi-directional workflow, the ability to automatically trigger events, integration with Slack collaboration software, the ability to encrypt data and Statuspage services integration to make it possible to share alerts with third-party services. Atlassian has also rebranded its Automation Actions to Opsgenie Actions; the company acquired Opsgenie last fall.

    Berkay Mollamustafaoglu, co-founder and head of product for Opsgenie, said now that Opsgenie is part of the larger Atlassian product family there’s clearly more focus on helping organizations integrate best DevOps practices with more structured processes based on the ITIL format. The latest version of the ITIL format provides makes those processes more flexible, but there’s still plenty of room for Atlassian to improve the tenor of the conversation across what has become a bi-modal IT operations environment. As the level of synchronization across those processes increases, the amount of fatigue IT operations teams experience lessens as the number of alerts that need to be correlated across multiple platforms decreases, Mollamustafaoglu noted. In fact, rather than convening in a physical “war room” to resolve a issue, most incidents will increasingly be resolved via online chats, he said.

    It may take a while before a period of détente emerges between advocates of DevOps and ITIL processes inevitably emerges. In the meantime, the more IT services management (ITSM) platforms become integrated with one another, the easier it becomes to achieve that goal. In fact, the major obstacle that needs to be overcome are much more likely to cultural rather than the level of integration that can be achieved using any specific ITSM platform.

    — Mike Vizard

  • New Relic Accelerates AIOps with SignifAI Acquisition

    New Relic Accelerates AIOps with SignifAI Acquisition

    New Relic, following its acquisition of SignifAI, is in the process of accelerating its investments in machine learning algorithms and other forms of artificial intelligence (AI) as part of an effort to make it easier to separate the signal from the noise created by all the alerts in an IT environment.

    Aaron Johnson, senior vice president of product management at New Relic, said SignifAI represents an opportunity for New Relic to create a superset platform that can apply advanced analytics to a much wider range of data sources. SignifAI has already created more than 60 integrations spanning open source and commercial monitoring tools that are widely used across DevOps toolchains, he noted, adding that as New Relic moves to add other data sources, there’s an opportunity for the company to distinguish itself beyond providing software-as-a-service (SaaS) tools for monitoring applications and infrastructure.

    It’s obviously still early days for AI in the realm of DevOps. But it’s also clear that with the rise of microservices and containers, soon it will be impracticable for IT organizations to manually manage their IT environments. Various forms of automation based on AIOps platforms infused with machine and deep learning algorithms will be required. As a provider of SaaS-based tools for monitoring applications and infrastructure, New Relic already has access to vast amounts of data required to train AI models on optimizing a specific IT environment.

    Of course, there’s lots of competition across the IT monitoring category, so it remains to be seen which vendors will have the resources required to drive continuing investments in AIOps.

    In the meantime, DevOps teams will need to adjust their processes to take advantage of AIOps. It’s unlikely AIOps platforms will eliminate the need for DevOps teams, but the number of applications and platforms any DevOps team must manage effectively will increase substantially. Rather than replacing humans, AIOps platforms will reduce the pressure to hire additional IT staff when professionals with DevOps skills are already in short supply. The AIOps platform, for all intents and purposes, acts like a co-pilot.

    The biggest issue IT organizations are likely to encounter along that journey is establishing trust with any AIOps platform they implement. It takes a while for most AI models to learn the IT environment before that platform makes any reliable recommendations. In most cases, DevOps teams will have to dedicate staff to train and update AI models continuously as the IT environment changes. It may take years, however, before DevOps teams are comfortable enough to move beyond trusting a recommendation and enable an AIOps platform to automate processes on its own. After all, the only thing worse about being wrong about any IT decision is being wrong about it at scale.

    AI at this point is all but inevitable in DevOps environments. But it doesn’t necessarily follow that just because something is based on AI it always will be smarter than experienced DevOps professionals.

    — Mike Vizard