devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Webb.ai Launches AI-Based Continuous Root Cause Analysis

    Webb.ai Launches AI-Based Continuous Root Cause Analysis

    Webb.ai today opened an early access program access to a Continuous Automated Root Cause Analysis platform that uses artificial intelligence (AI) combined with a knowledge graph to dynamically keep track of and analyze changes to IT environments.

    Manish Gupta, CEO of Webb.ai, said the platform uses a mix of machine learning algorithms and generative AI capabilities based on large language models (LLMs) to aggregate alerts into higher-order insights that are presented in natural language to members of a DevOps team.

    That approach differs from legacy approaches because it tracks causal relationships between changes versus relying on probabilistic models that tend to create a lot of false positives, he noted.

    Webb.ai is also taking advantage of Kubernetes clusters to enable its platform to scale up and down as needed when analyzing data, added Gupta. That platform is also more efficient than other approaches to observability that require IT teams to store massive amounts of data. In contrast, the Continuous Automated Root Cause Analysis automatically surfaces anomalous events that DevOps teams can choose to save rather than requiring them to save all the metrics, logs and traces that would otherwise need to be tracked, he noted.

    Most IT incidents can be traced back to the most recent changes made to an IT environment. The challenge is as IT environments have become more complex, it’s not feasible for humans to keep track of those changes without relying on AI and a knowledge graph, said Gupta.

    As businesses become more dependent on software to drive digital processes, the cost of downtime is exponentially increasing. The need to identify the root cause of an IT incident as fast as possible has become a critical requirement, he added.

    Webb.ai doesn’t see AI technologies replacing the need for DevOps professionals and site reliability engineers (SREs) for at least another 10 to 15 years. In the meantime, many tasks that today are completed manually will be increasingly automated, said Gupta. A DevOps engineer dragged out of bed at 2:00 A.M. to address an issue does not want to have to sift through alerts. Instead, generative AI tools will summarize the issue at hand with recommendations on resolving it, noted Gupta. In effect, DevOps teams will be augmented by AI so they can manage IT environments at even higher levels of scale, he said.

    There is always going to be some sense of fear and loathing when it comes to AI, but ultimately, AI should reduce the burnout rates that today adversely affects any organization’s ability to retain DevOps professionals. Far too much time is spent trying to identify the root cause of a slowdown in a cloud-native application’s performance that could be caused by any number of issues involving one dependency or another. The frustrating thing, of course, is that while it may take days to identify the root cause of an issue, it often takes just a few minutes to resolve.

  • The Role of AI in ITSM: SLAs, SLOs, SLIs and Error Budgets

    The Role of AI in ITSM: SLAs, SLOs, SLIs and Error Budgets

    In my recent blog Revolutionizing the Nine Pillars of SRE with AI-Engineered Tools, I classified SRE into nine pillars of SRE practices, a comprehensive framework that covers the full scope of SRE practices and suggested how AI can help with each of the pillars.

    For SLAs, SLOs, SLIs and error budgets—arguably the most important pillar of SRE—I indicated that AI can assist by providing accurate predictions of system performance and identifying potential bottlenecks. In this blog, I’ll explain in more detail how AI-engineered tools can be used to improve SLAs, SLOs, SLIs, error budgets and error budget policies within IT service management (ITSM).

    AI for SLAs, SLOs, SLIs, Error Budgets and Error Budget Policies

    Service Level Agreements (SLAs): An SLA is a contract between a service provider and the end user. It outlines the metrics by which the service is measured as well as remedies and penalties if agreed-upon service levels are not achieved. Setting the right SLAs is crucial, as they directly impact on customer satisfaction and the service provider’s reputation.

    The challenges in defining SLAs often involve understanding the trade-off between the cost of providing the service and the level of service that customers expect. AI can assist in this process by providing data-driven insights to inform these decisions. By analyzing historical performance data, customer usage patterns and feedback, AI models can predict the impact of different service levels on both customer satisfaction and operational costs. This can help service providers to set SLAs that meet customer expectations without unnecessarily inflating costs.

    AI can also be used to proactively monitor the agreed-upon SLAs. By continuously analyzing performance data, AI tools can detect potential SLA breaches before they occur, allowing the service provider to take preventative action. This kind of proactive SLA management can help to improve customer satisfaction and reduce the risk of penalties for SLA breaches.

    Service Level Objectives (SLOs): Establishing SLOs involves making informed predictions about system performance, defining realistic yet challenging targets that align with user expectations and business goals. A common challenge in defining SLOs is dealing with the complex nature of distributed systems and their interdependencies, making it difficult to predict performance and set appropriate targets. AI can assist in this process by analyzing historical data to model system behavior and predict future performance trends. This allows SRE teams to set more accurate, data-informed SLOs that reflect the system’s true capabilities. For instance, machine learning models could identify patterns in traffic and use this information to predict periods of high demand, allowing SRE teams to define SLOs that account for these peaks.

    Service Level Indicators (SLIs): SLIs are quantitative measures of some aspect of the level of service such as latency, throughput, availability, error rates, etc. The challenge lies in identifying the right metrics that accurately reflect the user’s experience and the system’s health. Traditionally, these decisions are based on domain knowledge, experience, and sometimes trial and error. AI can assist by analyzing a broad range of metrics across the system, correlating them with issues, incidents and user satisfaction levels to identify which indicators provide the most meaningful measure of service quality. By doing so, AI can help teams select the most appropriate SLIs, thus providing a more accurate picture of their system’s health and performance.

    Error Budgets: An error budget represents the acceptable level of risk or unreliability in a service, providing a balance between reliability and the pace of innovation. Determining an appropriate error budget involves an understanding of system behavior, user tolerance for errors, and business needs. The challenge lies in predicting future performance and understanding the likely impact of potential issues. AI can help overcome these challenges by simulating different scenarios and predicting their impact on the error budget. Machine learning algorithms can learn from past incidents and understand the correlations between different factors, helping teams predict future performance, establish an appropriate error budget, and make better-informed risk decisions.

    Error Budget Policies: These are policies that dictate actions based on the remaining error budget. These decisions often involve human judgment, like deciding when to freeze releases due to an exceeded error budget or when to allow more risk for a new feature rollout. AI can support these decisions by using predictive models to estimate the likely impact of different actions on the error budget. For example, it could simulate the impact of a new release on system reliability, providing insights to help decide whether to proceed with the release or hold it back. By doing so, AI can help teams manage their error budgets more effectively and make more data-driven decisions.

    Implementation Roadmap

    Implementing AI tools for SLAs, SLOs, SLIs, error budgets, and error budget policies must be well-planned and executed systematically. Here’s a practical implementation roadmap:

    Assess the Current Situation: Start by examining the current situation, identifying the areas of need and understanding the existing process of how SLAs, SLOs, SLIs and error budgets are being managed. Also, evaluate the data you currently have available and its quality because AI tools rely heavily on data.

    Define the Goals: Clearly outline what you hope to achieve with the implementation of AI tools. This could include better prediction of system performance, quicker identification of potential bottlenecks, more effective allocation of error budgets, etc.

    Select the Right Tools: Research and select suitable AI tools. Examples of AI-engineered tools that could be useful include:

    • Nobl9 for SLO management: It can assist with setting, monitoring and managing SLOs with AI-driven predictions.
    • Dynatrace: This tool uses AI to detect and diagnose system anomalies, improving observability and helping to inform SLOs and error budgets.
    • Datadog: Offers machine learning-based features that can help forecast future infrastructure metrics, contributing to better SLA and SLO management.
    • New Relic: Its applied intelligence feature can help with incident detection and resolution, helping to better manage error budgets.

    Data Preparation: Aggregate data from various sources, clean it and possibly format it in a way that’s suitable for the AI tools.

    Integration and Deployment: Integrate the AI tools with your existing systems to ensure that the AI tools can effectively communicate with your existing systems.

    Training and Calibration: Many AI tools require a period of training and calibration where the tool learns from your data to make accurate predictions and recommendations.

    Monitoring and Evaluation: Once the AI tools are in place, continuously monitor their performance.

    Iterate and Improve: Based on the evaluations, make necessary adjustments to the tools, your data or your goals.

    Training the Team: They need to understand how to interpret the results and how to take action based on the recommendations of these tools.

    Summary

    In the complex landscape of ITSM, AI is emerging as a transformative force, enabling organizations to manage their systems with unprecedented precision and proactivity. Through smart analysis of historical data and predictive modeling, AI tools are elevating how we define and handle SLAs, SLOs and SLIs. Gone are the days of generic, one-size-fits-all service-level targets. Today, AI empowers us to set realistic, data-informed goals that accurately reflect a system’s capabilities and align with user expectations, ensuring high customer satisfaction and operational efficiency.

    Furthermore, AI brings remarkable improvement to the management of Error Budgets and Error Budget Policies. By simulating different scenarios and predicting their impact, AI helps teams proactively manage risk, making well-informed decisions about error allowance and guiding actions when error budgets are threatened. AI enables us to strike the perfect balance between reliability and innovation, allowing IT teams to continue pushing boundaries without compromising service quality. As the era of AI in IT service management dawns, we stand on the cusp of a new phase of efficiency, control, and excellence in system management. So, join us as we delve deeper into this fascinating intersection of AI and SRE practices.

  • Splunk Adds Hub to Extend Observability to Edge Computing Platforms

    Splunk Adds Hub to Extend Observability to Edge Computing Platforms

    Splunk today launched a platform for ingesting data from sensors, industrial equipment and internet-of-things (IoT) platforms so that it can be analyzed alongside data collected from IT environments.

    Announced at the company’s .conf23 conference, the Splunk Edge Hub extends the reach of the troubleshooting platform Splunk provides that increasingly involves various types of edge computing platforms.

    Tom Casey, senior vice president for products and technology at Splunk, said Splunk Edge Hub is designed to be deployed in those edge computing environments and reduce the total amount of data that needs to be transferred back to an instance of the Spunk analytics platform running in a data center or in the cloud.

    That approach makes it possible to remotely monitor everything from environmental conditions to identifying anomalies indicative of an outage that might be prevented. The overall goal is to unify visibility across IT and operational technology (OT) environments as these systems and applications become more integrated, said Casey.

    It’s not clear how quickly that integration is occurring, given the historic cultural divide that has existed between IT and OT teams. But with the rise of digital business transformation initiatives, the data collected from edge computing platforms is increasingly being incorporated into a wide range of applications deployed at the edge, in local data centers and in the cloud.

    In general, observability tools are being used to not only improve processes but identify and block suspicious activity. Of course, OT teams have been monitoring edge computing platforms for decades, but observability platforms make it feasible to interrogate logs, metrics and traces to hopefully discover issues before there is a disruption to services. The challenge, of course, is that not everyone on either the OT or IT team necessarily knows how to shape the queries required to discover the root cause of a potential issue. The Splunk approach essentially leverages the skillset of IT professionals that have mastered its platform to address that issue; something most OT professionals are not going to be able to do using their existing tools and platforms.

    One way or another, DevOps teams will soon need to be able to remotely diagnose edge computing issues as the number of workloads deployed on these platforms expands. IT teams are not going to be able to travel to the physical location of every edge computing platform every time an issue arises. In many ways, the rise of edge computing will force a transition to observability tools that will enable IT teams to more proactively manage IT environments rather than responding after a metric that OT teams are monitoring has already exceeded its threshold.

    In the event there is still an issue, observability platforms should also reduce mean-time-to-remediation (MTTR) as visibility into IT environments, including edge computing platforms, is improved.

    It’s not likely that OT and IT teams will ever be completely unified, but as observability platforms continue to evolve, there will be much greater opportunities for collaboration.

  • Forking RHEL! Oracle and SUSE Join the Fight ¦ Silverman Sues AI Firms

    Forking RHEL! Oracle and SUSE Join the Fight ¦ Silverman Sues AI Firms

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: SUSE announces a Red Hat Enterprise Linux fork after Oracle said a similar thing, and Sarah Silverman says OpenAI and Meta have stolen her words. (more…)

  • JFrog Adds Curation Capability for Open Source Software Components

    JFrog Adds Curation Capability for Open Source Software Components

    JFrog today announced it is adding a curation capability for open source software to its portfolio that will use metadata generated by binaries to identify malicious packages and software components that have licensing issues.

    Sharan Hiremath, senior product manager for JFrog, said JFrog Curation is an extension of the JFrog Artifactory binary repository that the company provides to enable organizations to manage their software supply chains.

    JFrog Curation also validates software packages against JFrog’s Security Research library of recorded Critical Vulnerabilities and Exposures (CVEs) and publicly available information to help establish a trusted repository of pre-approved, third-party software components.

    That approach automatically employs the JFrog Xray scanning tool to eliminate the need to individually scan each package before developers employ them, said Hiremath.

    Making sure open source software components are secure is often problematic because developers download them from public repositories that are frequently targeted by cybercriminals. There have now been more than 100,000 instances of malicious software packages discovered in repositories in the last year alone, noted Hiremath.

    In addition, cybercriminals have become adept at creating fake repositories that mimic legitimate repositories; that can result in developers downloading software components infected with malware.

    JFrog Curation takes the shift left concept of application security to the next level by automatically blocking the use of risky open source software packages, noted Hiremath. The JFrog platform will also surface advice on how to remediate any vulnerability discovered, in addition to creating an audit trail to enable organizations to comply with various regulatory mandates.

    While open source software components reduce the cost of application development, from a cybersecurity perspective, it’s the wild, wild West, noted Hiremath. Organizations are trying to bring more discipline to the way software is constructed by adopting DevSecOps best practices, but they need an approach that doesn’t introduce any additional friction into the application development process, he added.

    JFrog Curation addresses that issue by addressing cybersecurity and compliance issues before any open source component becomes part of the build, said Hiremath.

    As regulations to hold organizations more accountable for the security of the software they deploy are being proposed around the world, it’s only a matter of time before organizations must revisit the security of their software supply chains. Cybercriminals have already proven how adept they are at injecting malware into software components that could be used in thousands of downstream applications. In the eyes of regulators, organizations that don’t scan open source software for malware and vulnerabilities are engaging in reckless behavior that will warrant stiff penalties.

    The challenge is that, after giving developers free rein to develop software any way they see fit, organizations are now faced with a challenge that is as much cultural as it is technical. Automated processes will naturally play a key role in curbing aberrant behavior in application development, but there is, as always, no substitute for some good old-fashioned common sense.

  • BigPanda Extends AIOps Platform to Embrace Generative AI

    BigPanda Extends AIOps Platform to Embrace Generative AI

    BigPanda today added generative artificial intelligence (AI) capabilities to analyze incidents surfaced by its AI for IT operations (AIOps) platform.

    Blair Sibille, field CTO for BigPanda, said Generative AI for Automated Incident Analysis leverages generative AI platforms based on the general-purpose large language models (LLMs) created by OpenAI.

    The goal is to make it simpler for IT teams to discern the root cause of an issue by, for example, taking advantage of summaries of alerts enabled by generative AI to make it simpler for IT professionals of any skill level to understand what requires their most immediate attention, he added. Today, insights are lost in the sea of alerts that often inundate IT teams, Sibille noted.

    BigPanda has been at the forefront of applying machine learning algorithms to automate IT operations management. Now, the company is incorporating external general-purpose LLMs to further augment IT teams, said Sibille.

    As IT environments continue to be more complex, it’s becoming essentially impossible for IT teams to manage them at scale without relying more on automation. AI technologies take automation to another level, but they won’t eliminate the need for DevOps teams to manage IT environments, added Sibille.

    BigPanda claimed generative AI capabilities will make it possible to reduce the amount of time required to resolve tickets by seven minutes each, with an accuracy rate of 95% during beta testing. Over time, those capabilities not only reduce mean-time-to-resolution but also reduce the level of toil that often conspires to burn out DevOps teams, said Sibille.

    At the core of the BigPanda AIOps platform is a data lake capable of consuming data generated by multiple applications and systems. That data then provides the foundation on which machine learning algorithms are applied to surface insights.

    It’s not clear how much generative AI will automate DevOps workflows, but in effect, more IT teams will be able to take advantage of having the equivalent of a DevOps engineer that they can query any time an issue arises using natural language, noted Sibille.

    AI doesn’t mean that IT outages will be completely eliminated, but it does mean they will become a lot less stressful, he added.

    Naturally, there’s a lot of speculation concerning the impact AI will have on DevOps. Some contend it’s only a matter of time before the entire software engineering process is automated. As those advances are made, the cost of building applications will effectively drop to zero. Others argue that AI will simply augment DevOps teams in a way that makes it simpler to manage IT at scale using either the same number or possibly even fewer engineers.

    The one thing that is clear is that AI will soon be pervasive. DevOps teams should start evaluating today which tasks will be automated with an eye toward planning how DevOps roles will evolve as AI advances accelerate.

  • Roadie Adds Scorecard Tool to Backstage SaaS Platform

    Roadie Adds Scorecard Tool to Backstage SaaS Platform

    Roadie today made generally available a tool that automatically tracks and reports how software development teams are performing against engineering standards defined by a DevOps team. The Tech Insights tool is an addition to Roadie’s implementation of the open source Backstage developer portal and is delivered via a software-as-a-service (SaaS) platform.

    Roadie CEO David Tuite said Tech Insights is a proprietary tool that Roadie developed as an extension to its platform. Tech Insights provides scorecards that track software assets in the Backstage catalog and ensure they are meeting predefined quality and compliance targets.

    Tech Insights comes with built-in, pre-defined data sources; alternatively, DevOps teams can define them using external application programming interfaces (APIs) or files in a repository. A rule known as a Check can then be created to validate whether a project complies with the engineering standards defined by the DevOps team. Each DevOps team can create an unlimited number of scorecards.

    In addition, teams can create prompts and notifications to warn engineers when they are about to make a change that will violate an engineering standard. DevOps teams can also employ gamification techniques to show teams how their performance compares to others.

    While Roadie is now adding its first proprietary tool to the platform, Tuite said the company remains committed to enabling organizations to migrate their instance of Backstage to another environment should they ever deem it necessary, he added.

    Originally developed by Spotify, Backstage makes it possible to create and manage a catalog of blueprints that developers can readily consume rather than requiring them to repeatedly build these capabilities themselves. The goal is to create scaffolds that developers can consistently reuse across multiple application development projects.

    Spotify has since donated Backstage to the Cloud Native Computing Foundation (CNCF). However, Backstage currently requires a significant amount of DevOps expertise to set up and maintain, an issue that Roadie is addressing by making it available as a SaaS platform that can be accessed by any application development team, said Tuite.

    The concept of a self-service portal through which developers can address their own needs may not be new, but as an open source project, Backstage is lowering the cost of improving overall developer productivity. At the same time, many organizations are now embracing platform engineering as a means of centralizing the management of DevOps.

    Ultimately, every minute developers spend on tasks other than writing code is one less minute they are spending writing the business logic that drives an application. Measuring developer productivity is, of course, an inexact science. No one knows for sure when inspiration might strike, but if developers are busy maintaining code that they created to provision infrastructure, it’s less likely they’ll be struck by a flash of brilliant insight that improves and application. The easier it is to spin up a development environment, the more likely that inspiration will one day turn into code that runs in an actual production environment.

  • Loft Labs Makes Tool for Creating Dev Environments Open Source

    Loft Labs Makes Tool for Creating Dev Environments Open Source

    Loft Labs has made available an open source software project that enables DevOps teams to programmatically create developer environments using a tool that runs entirely on a local client.

    Lukas Gentele, Loft Labs’ CEO, said DevPod provides an alternative to cloud-based approaches such as Codespaces from GitHub that lock organizations into a specific platform for building applications.

    Instead, DevPod can automatically spin up developer environments as code on any public cloud or on-premises IT environment that can run a devcontainer.json file, he added.

    In addition, DevPod will also automatically put developer environments to sleep when not being used to reduce costs, noted Gentele. Other capabilities include support for prebuilds, Git and Docker credential synchronization.

    DevPod was created by Loft Labs to streamline the development of a virtualization platform for Kubernetes clusters. The company is now making DevPod available as open source software that any DevOps team can use to manage developer environments as code more easily than complex server-side alternatives, said Gentele.

    It also makes it easier for developers to switch between cloud-hosted and local environments as needed seamlessly, he noted. That’s critical because it enables organizations to reduce dependencies on cloud computing environments for writing code by allowing developers to use their local machine whenever they prefer, noted Gentele.

    Despite the ubiquitous availability of cloud services, most developers continue to prefer to write code on a local machine. The challenge is that replicating a developer environment elsewhere to foster collaboration has been problematic. DevPod provides a means for developers to collaborate with each other and with software engineers without giving up the inherent flexibility a local development platform provides.

    At a time when organizations are more dependent on software than ever, there is much more focus on developer productivity. DevOps teams have historically tended to focus on DevOps Research and Assessment (DORA) metrics that focus more on deployment frequency and mean-time-to-recovery than they do on developer productivity.

    Of course, measuring developer productivity is an inexact science. An organization might once have tracked lines of code written to gauge productivity, but that is not especially good metric. In more recent times, metrics have been developed that, among other things, keep track of how much time developers spend actually coding versus maintaining their software development environment.

    The issue that most IT leaders will immediately encounter is that every minute developers spend on these tasks, however, is one less minute they spend writing the business logic that drives an application.

    Obviously, there’s no way developers are going to write business logic daily for eight hours straight; there are maintenance windows. However, application development is as much art as it is science. No one knows for sure when inspiration might strike, but if developers are busy maintaining code created to provision infrastructure, it’s less likely they’ll have a flash of brilliant insight that allows them to make an application better. The easier it is to spin up a development environment, the more likely it becomes that inspiration will one day turn into code that runs in an actual production environment.

  • Revolutionizing the Nine Pillars of SRE With AI-Engineered Tools

    Revolutionizing the Nine Pillars of SRE With AI-Engineered Tools

    In my blog Rapid Strategic SRE Assessments Accelerate IT Transformations published last year, I classified site reliability engineering (SRE) into nine pillars of SRE practices—a comprehensive framework that covers the full scope of SRE practices. This nine pillars SRE framework is consistent with the SRE blueprint published by the DevOps Institute. Each of these pillars represents a vital area of SRE implementation and, together, they ensure a holistic approach to aligning development and operations for effective collaboration, uneventful continuous deliveries and reliable, highly scalable, safe operations.

    The current generation of AI-engineered tools is now being applied to each of the SRE pillars and has the potential to significantly increase efficiency, reliability and scalability for all SRE practices.

    Just as SRE practices complement DevOps practices, these nine pillars of SRE complement the nine pillars of DevOps explained in my recent blog Revolutionizing the Nine Pillars of DevOps With AI-Engineered Tools.

    AI Applications for the Nine Pillars of SRE

    The following briefly explains how the current generation of AI-engineered large language model (LLM) tools are being applied or could be practically applied to improve each of the nine pillars of SRE.

    Culture: AI can help identify patterns and trends in system behavior that may not be immediately evident. This can assist in shifting the wisdom of production further to the left. Tools like Datadog’s Watchdog can be used to automatically detect and surface anomalies in your system behavior, for example.

    Toil Reduction and Automation: AI can help reduce toil by automating more complex tasks that were previously challenging to automate. Machine learning models can predict system behavior, enabling proactive responses. AI operations platforms like Moogsoft or BigPanda can automate incident management, from detection to remediation.

    SLAs/SLOs/SLIs and Error Budgets: AI can assist in refining SLAs, SLOs and SLIs by providing accurate predictions of system performance and identifying potential bottlenecks. Tools like Nobl9 can use machine learning to better predict and define SLOs based on historical performance data.

    Measurements and Observability: AI can help analyze vast amounts of data from monitoring and observability systems, identifying patterns and correlations that may be difficult for humans to detect. Tools like Dynatrace and New Relic use AI to analyze monitoring data, detect anomalies and correlate events.

    Anti-Fragility: AI can contribute to the development of more resilient systems by analyzing system behavior under stress and identifying vulnerabilities. The data gathered from chaos experiments using tools like Gremlin can be used in conjunction with machine learning models to predict system behavior.

    Work-Sharing and Incremental Technical Debt: AI can help analyze code and infrastructure changes to assess their impact on technical debt. Tools like DeepCode and SonarQube use AI to provide guidance on how to manage this debt incrementally.

    Deployments: AI can be used to optimize deployment strategies by predicting the impact of new releases. A tool like Harness can use machine learning to optimize deployment strategies and minimize risks.

    Performance Management of Apps and Infrastructure: AI can analyze application and infrastructure performance data to identify optimization opportunities and predict capacity requirements. Tools like Turbonomic leverage AI to optimize resource allocation in real-time.

    Incident Management: AI can assist with incident management by automating the detection and triage of incidents, helping to quickly identify the root cause. Tools like PagerDuty use AI to automate detection, triage and suggest potential remediation steps, leading to faster resolution times.

    Pitfalls and Challenges

    Applying AI to SRE is a complex process with certain challenges. Here are some potential pitfalls along with ways to address them:

    Lack of Quality Data: AI and machine learning models are only as good as the data they are trained on. Inadequate or poor quality data can lead to inaccurate predictions and insights.
    • Prioritize data hygiene and governance. Collect comprehensive and diverse data from your systems; ensure that it is well-structured and free of errors and store it in a way that’s easily accessible for training AI models.

    Over-reliance on Automation: While AI can greatly enhance automation, relying on it too heavily without human oversight can lead to missed signals or overcorrections in response to false positives.
    • Maintain a balance between automation and human oversight. Use AI to support decision-making, not replace it entirely. It’s important to have experienced SREs review AI outputs regularly to ensure they make sense and are beneficial.

    Underestimating the Need for AI Expertise: Implementing AI is not just about buying and deploying a tool. It requires a deep understanding of AI and machine learning principles, as well as the ability to interpret the results correctly.
    • Invest in training your team on AI and machine learning principles or consider hiring AI specialists. Having in-house AI expertise can help ensure that you’re using AI tools effectively and interpreting their outputs correctly.

    Ignoring the Importance of Integration: AI tools need to work seamlessly with your existing infrastructure and processes to be effective. A lack of integration can lead to data silos and inefficiencies.
    • When selecting AI tools, consider how easily they can be integrated into your existing infrastructure and workflows. Use APIs and other integration methods to ensure that your AI tools can access the data they need and that their insights are easily accessible to your SRE team.

    Neglecting Ethics and Privacy: AI tools often need access to sensitive data to operate effectively, which can raise ethical and privacy concerns.
    • Implement robust data privacy and security measures. Ensure that you’re following all relevant regulations and industry best practices for data privacy. Use anonymization and other data protection techniques to protect sensitive data.

    Summary

    Artificial intelligence is revolutionizing SRE practices by automating complex tasks, analyzing vast amounts of data and making proactive predictions. AI can reduce toil, enhance system understanding, and streamline incident management. It allows SREs to shift further left by integrating predictive insights into the development process, thereby enhancing the culture of proactive issue prevention. AI also aids in defining more precise service level objectives, error budgets, and optimizing deployment strategies based on predictive analytics. In the realm of incident management, AI aids in detecting, triaging, and mitigating incidents faster by identifying patterns and suggesting remediations. Overall, AI is fast becoming an essential tool for SREs, facilitating a shift from reactive problem-solving to proactive management, thereby increasing system reliability, performance, scalability and efficiency.

  • Threads: Twitter Killer or Ad-Infested Hellscape?

    Threads: Twitter Killer or Ad-Infested Hellscape?

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Meta launches its much-leaked Threads microblogging app. A spinoff from Instagram, it’s already reached 30 million signups at the time of writing—even before it’s available in the EU. But how is Meta’s DevOps team making it scale? (more…)