devopsy.kinsta.cloud

Tag: ai

  • Ciroos.AI Preps AI SRE Agents Trained to Automate Incident Management

    Ciroos.AI Preps AI SRE Agents Trained to Automate Incident Management

    Ciroos.AI this week emerged from stealth to provide early access to a set of artificial intelligence (AI) agents that have been trained to augment site reliability engineers (SREs).

    Fresh off raising $21 million in additional funding, Ciroos.AI CEO Ronak Desai said the SRE Teammate provides access to multiple extensible AI agents that have been trained to proactively investigate anomalies without necessarily having to be directed.

    There is already a chronic shortage of SREs, so any tasks that can be assigned to AI agents is only going to reduce the stress level of existing teams, noted Desai. AI agents, for example, are already capable of reducing incident response times by up to 90% to enable SRE, he added.

    That capability will prove critical as the overall volume of applications deployed in production environments continues to steadily increase in the age of AI, noted Desai.

    The SRE Teammate platform makes use of the Model Context Protocol (MCP) developed by Anthropic and Agent2Agent (A2A) architecture defined by Google to enable AI agents to interact with each other.

    Ultimately, DevOps teams will soon consist of multiple autonomous agents that are supervised by software engineers to ensure tasks are completed as expected, said Desai.

    It’s not clear how quickly DevOps teams are embracing AI to manage workflows, but Futurum Group research finds 41% of survey respondents expect generative AI tools and platforms will be used to generate, review and test code.

    Even less apparent is the extent to which organizations will adopt AI copilots and agents that have been added to their existing platforms versus opting to acquire new platforms developed from the ground up for AI agents that are supervised by software engineers.

    Regardless of approach, many of the manual tasks that today conspire to make DevOps workflows more tedious will be eliminated. There is no doubt that will also change the role of software engineers, but the primary goal remains finding ways to leverage automation to enable organizations to deploy more applications in production environments that can run at higher levels of scale.

    Additionally, AI technologies should make DevOps tools and platforms more accessible to a wide range of organizations that previously could not afford the level of investment required.

    Each organization will, of course, need to determine the level of faith they are willing to place in AI. The challenge is that, given the current pace of innovation, the capabilities of an AI tool that might be made available in a few short months will be substantially higher than the current iteration. As such, DevOps teams need to base their overall strategy not just on what AI tools can do today but, more importantly, what kinds of capabilities will be available in the near future.

    In the meantime, DevOps teams would be well-advised to start making a list of tasks that are likely to be assigned to an AI agent with an eye toward determining how the role of software engineers will evolve from there.

  • This Bankrupt AI Startup Was More Artificial Than Intelligent

    This Bankrupt AI Startup Was More Artificial Than Intelligent

    It turns out Builder.ai was more artificial than intelligent.

    The $1.5 billion artificial intelligence (AI) company backed by Microsoft Corp. recently shuttered operations after its neural network was exposed to actually be 700 engineers in India.

    Builder.ai, which has begun formal bankruptcy proceedings in jurisdictions where it operated, including India, the U.K., and the U.S., said on LinkedIn it would be “entering into insolvency proceedings.”

    “Despite the tireless efforts of our current team and exploring every possible option, the business has been unable to recover from historic challenges and past decisions that placed significant strain on its financial position,” the company wrote.

    The company’s Natasha, an AI app-building service, helped it land a strategic collaboration and “significant” investment with Microsoft two years ago. At the time, Builder.ai CEO Sachin Dev Duggal said, “From my first meeting with Microsoft to the moment we agreed to collaborate more strategically, one thing has been really clear — Microsoft’s commitment to helping everyone unlock their true potential.”

    Documents reviewed by Bloomberg showed Builder worked with VerSe, an India-based social media startup, to falsely increase its sales numbers, regularly billing each other for similar amounts between 2021 and 2024. “We’re not the kind of company that is in the business of inflating revenues,” VerSe co-founder Umang Bedi told Bloomberg, calling the accusations “baseless and false.”

    The sudden rise and fall of Builder.ai serves as a cautionary tale in an industry overflowing in hype, and one susceptible at times to scams. Since it was founded in 2016, the company originally known as Engineering.ai has been embroiled in scandal. It promised to make software creation “as easy as ordering pizza” and marketed its AI assistant, Natasha, as a breakthrough in building software with minimal human input.

    At its peak, Builder.ai raised more than $450 million from the likes of Microsoft, SoftBank’s DeepCore, and IFC to reach a valuation of $1.5 billion.

    But there were cracks in the corporate facade.

    Employees and outsiders pointed out Builder.ai relied on about 700 engineers in India who manually wrote code for client projects while the company misrepresented the work as AI-generated.

    Former employee Robert Holdheim sued the company for $5 million, alleging he was fired after flagging concerns over deceptive practices. Legal filings subsequently revealed Builder.ai misled investors by claiming apps were “80% built” by AI though the supporting tech was barely functional. A 2019 investigation by the Wall Street Journal found most of the coding was done manually.

    And the saga of Builder.ai is far from over: The company reportedly owes $85 million to Amazon.com Inc. and $30 million to Microsoft in unpaid cloud services. A U.S. federal probe is underway, with investigators seeking access to its financial and client data.

  • Managing Day 2 Concerns for Agentic AI Architecture

    Managing Day 2 Concerns for Agentic AI Architecture

    We’re now in the fifth wave of computing. If that sounds dramatic, that’s because it is—and probably also because someone in marketing really wants to make “agentic AI” stick. For those keeping track at home, the first four waves were mainframes (your granddad’s computer), web (when we were all hopeful), cloud (when we were still kind of hopeful), and mobile (where that hope became a subscription to your attention span).

    And now? Agentic AI.

    Let’s get the buzzwords out of the way: Agentic AI is a fancy term for giving large language models (LLMs) persistent memory, looping logic, and callbacks to reach out to your APIs. It’s like giving a Roomba a graduate degree and asking it to run your IT department. These “agents” operate independently (ish), have goals (ish), and make decisions (ish) based on prompts, semantic search and however much context you can cram into a vector database.

    Sounds fantastic. Until you have to operate and maintain this stuff in production.

    Because Day 1 is always a party, demos run great on slide decks. Day 2 is when the ops team walks into the room, sees the cloud bill, and quietly walks back out.

    The Scale Problem (a.k.a. We Broke TPS)

    Let’s talk about scale. Traditional systems had thousands to millions of users. With mobile, we got billions. Agentic AI? Try trillions. Not just people, though. You’ll have agents helping users, agents helping agents and agents judging the work done by other agents. 

    Mainframe

    Web

    Cloud

    Mobile

    Agentic

    Users

    thousands

    millions

    100 million

    billions

    trillions

    TPS

    100

    500

    2,500

    10,000

    1,000,000

    5x

    5x

    4x

    100x

    Each one of those agents runs loops. Each loop makes calls. Each call hits an LLM. Every LLM call contributes to Transactions Per Second (TPS). Not all transactions or LLM interactions are the same. Humans have very different behavior than agents.

    The types of interaction behavior we might see:

    • Human users = finite and sleepy
    • Agents = infinite and caffeinated
    • LLM inference = expensive, slow, and shockingly unreliable
    • Therefore: system load = Who knows? How do we even predict this stuff?

    It isn’t unusual for pre-agentic systems to max out around 10,000 TPS. Agentic systems? You’re looking at a million TPS just to hit parity with your company’s Slack usage.

    But What About Cost?

    Ah yes, the CFO’s favorite section.

    Running agents is expensive. Running smart agents is very expensive. Running smart agents 24/7 with high-accuracy LLMs? Welcome to a world where your annual AI ops bill might be an order of magnitude higher than it was before.

    Let’s put that in relatable terms:

    • GPT-4 32K (86% accuracy): $98 per million tokens
    • Open source model (42% accuracy): $0.013 per million tokens
    • Database transaction: pocket change by comparison

    And don’t let the open-source optimism fool you. A 42% accurate agent might write emails like “Dear Sir/Madman, I have eaten the spreadsheet,” which is cute until it’s running your treasury. Remember that hallucinations and bad responses are so common, we already have patterns for using multiple other LLMs just to judge the work of the first. This means that your cost isn’t just the cost to ask an LLM an innocent question, it’s the cost to resolve tools, cost to make the actual request and cost to make potentially multiple judgment requests.

    So now you’re faced with the AI version of the eternal tradeoff: dumb and cheap vs. smart and broke.

    Latency, Reliability and Other Sad Stories

    LLMs are not built for low-latency, high-throughput work. They’re built for… storytelling. And hallucinating. And sometimes – if you ask nicely and are an expert in prompt engineering – even answering your questions.

    Now, imagine your core systems relying on that. Imagine your uptime metrics depending on the whims of an LLM inference API that thinks “return customer data” means “find a retail location where customers can bring and return their data”

    Agentic systems aren’t transactional. They’re conversational. They’re unpredictable. They can fail to give you an answer, they can stop answering in the middle of a stream. Don’t listen to what people tell you, you can’t blindly trust an LLM.

    And yet, we keep trying to run them like traditional software. We build orchestration platforms, routing layers, retry loops, and cost governors. Why? Because otherwise, these things will try to eat the entire internet in a single for-loop and send us the cloud bill.

    Capacity Planning Is Now Chaos Planning

    You used to think about load in terms of peak usage. Now? Your agents don’t sleep. They don’t go to lunch. They just keep going, looping away, slamming vector DBs and semantic search engines with relentless, existential enthusiasm. Your system lows are higher, your median times are higher, and your peaks are higher.

    This means:

    • Your database load goes up 100x
    • Your network saturates under constant agent chatter
    • Your infrastructure team turns into a stress-testing experiment
    • And you still get worse answers than just asking the intern

    You don’t plan for “peak load” anymore. You plan for “non-stop Reddit debates between bots.”

    The Blueprint (Such as It Is)

    In theory, building a sustainable agentic architecture looks like this:

    1. Semantic Search – All your domain knowledge gets shoved into a vector DB, ready to be retrieved by semantic similarity.
    2. Agentic Sessions – Every agent interaction is logged to provide context, continuity and accountability.
    3. LLM Invocation Loops – Prompt, retrieve, augment, call LLM, process, repeat until you get a satisfactory answer, or you run out of usage quota.
    4. Orchestration Layer – The puppet master that tells agents what to do, when, and for how long before they bankrupt you.

    This is all manageable, assuming you can hire a team of distributed systems engineers who also speak fluent “prompt engineering,” understand vector math, and have no plans for sleep.

    What Ops Teams Need to Do (Besides Cry)

    If you’re serious about building something in this space—and not just gluing ChatGPT to your CI pipeline, you need to:

    • Plan for real-time TPS: Millions of them, all at once, with unpredictable spikes.
    • Plan for augmentation (RAG): Simple one-off queries are rare. Instead, your agents will be making multiple round trips to the LLM to resolve tools and multiple passes at a vector database for context augmentation.
    • Instrument your agents: Know which model they’re using, how often, and at what cost.
    • Design for failure: Assume models will hallucinate, APIs will fail and your agents will suddenly think they’re writing sci-fi.
    • Stay modular: You’ll want to swap models faster than your CFO can say “cost optimization.”

    Final Thought: The AI Agency Gap

    The AI Agency Gap is the chasm between dumb bots and actual autonomous systems. It’s the space where most agents will live—semi-capable, semi-stable, semi-helpful. They’re the interns of the software world: energetic, inconsistent and always one bad prompt away from disaster.

    If you want to thrive in this new wave of computing, you’re going to need resilience, observability, guard rails, patience and some really good stress coping techniques. The future isn’t just about building agentic systems, it’s about knowing that you’ve made all of your microservices unpredictable and require 10x more resources.

    Good luck.

  • Survey Surfaces Uneven Adoption of SBOMs to Secure Software

    Survey Surfaces Uneven Adoption of SBOMs to Secure Software

    A survey of 100 security professionals finds nearly half (48%) of security professionals admit their organizations are falling behind on meeting software bill material (SBOM) requirements as specified by the U.S. Office of Management and Budget (OMB) Memo M-22-18, Executive Order 14028, and the European Union (EU) Cyber Resilience Act.

    Conducted during the recent RSAC conference by Lineaje, a provider of tools and services for securing software supply chains, the survey also finds a nearly equal percentage (47%) have either not started SBOM integration or are still evaluating tools and practices.

    Additionally, just over a third conceded their organization still has difficulty identifying and tracking open-source components in applications accurately, while nearly a third (29%) still lack the tools and processes needed to analyze SBOMs for vulnerabilities.

    As a result, only 38% of respondents said their organization prioritizes fixing the most vulnerable areas within their applications,

    On the plus side, a full (88%) of respondents expect artificial intelligence (AI) to have the potential to critically or significantly enhance software supply chain security visibility. At the same time, respondents are also aware that AI also introduces additional data security and privacy risks (35%) as well as potential vulnerabilities when used to write code (26%).

    Lineaje CISO Nick Mistry said AI should also make it simpler to determine whether a fix is available for a vulnerability. However, while there seems to be a lot of confidence being placed in the ability of AI to improve software supply chain security, there is still much work to be done, he added. In fact, it’s already been shown how ChatGPT can be used to create an exploit for 87% of known vulnerabilities, so the number of vulnerabilities that might be exploited is about to substantially increase, noted Mistry.

    AI tools have also experienced hallucinations that result in references to software packages that don’t exist. Cybercriminals are now creating software packages that bear the same name as part of an effort to fool developers into downloading software packages that have malicious code, said Mistry.

    Despite those concerns, however, just under a third (32%) of survey respondents said they believe their organization will one day deliver software that has no vulnerabilities. In contrast, 68% said they are uncertain that goal might ever be achieved.

    Ultimately, the ability to build and deploy secure applications comes down to processes. Unfortunately, most of the processes being used to identify vulnerabilities and then mitigate them are still flawed, said Mistry. There is still a tendency to create long lists of vulnerabilities that are then thrown over the proverbial wall for application developers to investigate. Much of that effort, however, is wasted when it is determined that an application that might have a vulnerability is either not externally accessible or the code suspected of having a vulnerability was never actually loaded into memory.

    More troubling still, 70% of the survey respondents admit that when a fix is not available for a vulnerability, they either don’t have or are not sure if they have an alternative remediation plan in place.

    It’s clear that organizations need a more unified view of vulnerabilities that will enable cybersecurity and application developers to work more collaboratively, noted Mistry. The challenge and the opportunity now is to provide those insights into software supply chain security in a way that enables everyone involved to see the same issues not just the same way, but also at the same time.

  • Harness Adds MCP Server to Expose Data to Third-Party AI Tools

    Harness Adds MCP Server to Expose Data to Third-Party AI Tools

    Harness today revealed it has developed a model context protocol (MCP) server through which it will be able to provide more context to artificial intelligence (AI) tools, agents and platforms that are being used to build software.

    Rohan Gupta, a principal product manager for continuous delivery and AI DevOps for Harness, said exposing the data residing in the Harness continuous integration/continuous delivery (CI/CD) platform will streamline workflows by eliminating the need to rely on custom plug-ins and application programming interfaces (APIs) to provide additional context about the application development environments where code is ultimately deployed.

    Originally developed by Anthropic, MCP is based on a JSON-RPC 2.0 API framework and is rapidly becoming a de facto API standard for integrating AI tools, agents and platforms. The overall goal is to make it easier to expose relevant data to any AI agent that has permission to access it. The overall goal is to make it simpler for AI coding tools to more easily understand the reasons why, for example, a specific DevOps pipeline might have failed, said Gupta.

    DevOps teams will also be able to employ the role-based access controls (RBAC) provided by Harness to limit access to deployment logs only. All the API keys are managed through Harness and no data is ever sent to a large language model (LLM).

    It’s not clear how widely AI tools and platforms have been adopted but a Futurum Group survey finds 41% of respondents expect AI tools and platforms will be used to generate, review and test code, while 39% plan to make use of AI models based on machine learning algorithms.

    Ultimately, DevOps engineers will soon find themselves supervising a small army of AI agents with the help of what are becoming known as Planner agents, said Gupta. It remains to be seen how multiple AI agents will interact with one another to complete an end-to-end task but as orchestration frameworks continue to evolve there will be higher levels of interoperability between agents that both Harness and its third-party partners are continuing to build, said Gupta.

    In the meantime, DevOps teams might want to start creating an inventory of manual tasks that might soon be assigned to an AI agent. As the ability of these AI agents to reliably complete these tasks improves, DevOps engineers should be able to spend more of their time on activities that provide higher value to the business, he noted.

    Arguably, the next major challenge will be the cultural changes that will inevitably occur as more DevOps tasks are automated using AI agents that, for all intents and purposes, are now part of a DevOps team, added Gupta.

    Regardless of approach to building software, the expectation is that the data that AI agents need to write better code will be readily made available. Each organization will need to decide to what degree to trust AI technologies to build software themselves, but ultimately, as the pace of software innovation starts to accelerate in the age of AI, it may soon not be feasible to effectively compete without it.