devopsy.kinsta.cloud

Category: AI

  • Atlassian Adds CLI Option to AI Agent for Building Software

    Atlassian Adds CLI Option to AI Agent for Building Software

    Atlassian has added a command line interface (CLI) option to provide application developers with a familiar construct for invoking Rovo Dev, its artificial intelligence (AI) agent for building software

    Part of a portfolio of Rovo Software Agents that Atlassian now makes available in beta, Rovo Dev completes and debugs code, creates tests, surfaces insights and explanations of code bases, makes suggestions to improve code and generates documentation. It is also integrated with Jira project management software and Confluence collaboration tools provided by Atlassian.

    Using a set of software engineering (SWE) benchmarks maintained by researchers at Princeton and Stanford, Rovo Dev achieved a 41.98% resolve rate across 2,294 tasks.

    It can also be integrated via a Model Context Protocol (MCP) Server to access external data sources capable of providing more context that might be needed to complete a specific task.

    Shuyin Zhao, vice president of product for Atlassian, said the goal is to provide a single agent capable of performing a wide range of tasks versus requiring developers to create workflows that require them to orchestrate multiple agents to build and deploy software. The more memory that can be allocated to an AI agent the more it is able to preserve context, she added.

    Longer term, as the reasoning capabilities provided by large language models (LLMs) continue to expand, the scope of complex tasks that can be assigned to that AI agent will also increase, noted Zhao.

    Eventually, many developers will find they will be spending more time reading and evaluating code rather than writing it themselves, she added.

    It’s not clear how widely AI tools are being adopted by software developers and engineers, but a Futurum Group survey found 41% of respondents expect generative AI tools and platforms will be used to generate, review and test code. The one thing that is certain is more developers than ever are using these tools to write code, but how much of that code is making it into production environments is unknown. In some instances, an AI tool might enable a developer to write better code than they can on their own. In other instances, it might create vulnerabilities or recommend using software packages that simply don’t exist.

    Additionally, even as more code is generated, the existing DevOps pipelines might not be robust enough to deploy software at higher levels of scale than what is already being achieved.

    None of that means AI agents won’t be incorporated into DevOps workflows, but it does mean there is still a considerable amount of effort required to re-engineer those processes in a way that is optimized for AI agents rather than human application developers.

    In the meantime, however, DevOps teams would be well-advised to start creating a list of tasks they are willing to delegate to AI agents, assuming there is some level of supervision. Otherwise, it’s all but inevitable that AI agents might generate the same issues that human developers create, albeit just at a different level of scale. The issue then becomes making sure the right governance policies are in place to prevent as many of those issues from arising in the first place.

  • New Relic Adds Support for Model Context Protocol to Observability Platform

    New Relic Adds Support for Model Context Protocol to Observability Platform

    New Relic this week added support for the Model Context Protocol (MCP) to its observability platform to surface insights into artificial intelligence (AI) agents and applications.

    Originally developed by Anthropic, MCP is rapidly becoming a de facto application programming interface (API) for enabling interoperability between AI agents and other sources of data.

    New Relic has now incorporated MCP with its application performance monitoring to surface insights into AI agents and applications alongside its ability to monitor and observe legacy applications.

    Previously, New Relic had added an ability to observe large language models (LLMs), but via MCP it’s now able to extend the reach of that capability to AI agents and applications that access those LLMs. That capability makes it possible to visualize specific usage and patterns of the entire lifecycle of an MCP request, including invoked tools, call sequences and execution durations.

    Additionally, DevOps teams can identify which prompts are being invoked and what choices were made by an AI agent along with surfacing latency, errors and other MCP performance issues.

    New Relic AI Monitoring MCP support is now available in Python Agent version 10.13.0, with additional languages coming soon.

    In general, New Relic reports it has seen a steady 30% growth in usage of its AI monitoring tools quarter-over-quarter for the past year. The company has also seen a 92% increase in the number of unique AI models being used. However, the ChatGPT models provided by OpenAI accounted for 86% of all the LLM tokens being generated.

    Nic Benders, chief technical strategist for New Relic, said that level of activity suggests that OpenAI is well on its way to becoming a de facto standard, even as the number of AI models being made available continues to rapidly increase.

    It’s not immediately clear why one LLM might be preferred over another but OpenAI, in addition to making a steady stream of investments, has successfully generated a significant amount of brand recognition. That helps make OpenAI one of the first platforms that most organizations are going to look to when building an AI application, said Benders.

    However, there are use cases, such as coding, that lend themselves, for the moment at least, better to coding, he added.

    Less clear is the degree to which the various AI models that are available will gain enough traction to warrant the ongoing level of investment required to sustain them. IT teams should carefully consider what foundations they want to build and deploy AI applications on for the long term, noted Benders.

    Additionally, not every AI application requires the latest, and most expensive, LLM, said Benders. The cost of previous generations of AI models has declined considerably as more advanced models have become available. Many applications, however, may not require access to a more advanced model, especially when a previous generation will enable a developer to automate a task at a much lower cost.

    Ultimately, observability platforms will play a critical role in enabling organizations to track which AI models are being used for which use cases. The challenge now is understanding not only which ones are gaining traction among application developers, but just as importantly, the symbiotic relationship that exists between AI models and the IT infrastructure upon which they depend.