Tag: generative AI
-
Google Adds App Modernization Tools to Generative AI Platform
Google this week added automated refactoring capabilities to the generative artificial intelligence (AI) tools it makes available to write code on Google Cloud.
In addition, Duet AI for Google Cloud is now integrated with Google’s Apigee API Management and Application Integration services. This makes it possible to design, create and publish application programming interfaces (APIs) using simple natural language prompts.
These additions to the preview of Duet AI for Google Cloud were announced at the Google Cloud Next 2023 conference and are scheduled to be generally available later this year. The new capabilities promise to reduce the time and effort required to modernize applications.
For example, DevOps teams can now convert code written in C++ into Go and migrate that application to Cloud SQL, the managed relational database service that Google provides. Via a natural language interface, these tasks are launched via a prompt from directly within a development environment.
Priyanka Vergadia, a developer advocate for Google, told attendees these capabilities will reduce the need to rely on consultants to migrate applications and take up much less time. In addition, the overall quality of the code should improve using large language models (LLMs) that have been trained using code written by Google developers and best practices defined by Google software engineers.
Organizations will save time and effort because the need for developers to understand how existing application code was constructed is eliminated, she noted.
Google has also started to work with select enterprise IT teams to allow Duet AI to incorporate knowledge from their libraries and codebases to generate more context-aware code suggestions.
In addition to writing code, Google is making a case for also using Duet AI in Google Cloud to monitor performance and troubleshoot IT issues by identifying correlations across application environments. For example, natural language prompts can be translated into PromQL queries to analyze time-series metrics. Duet AI can also provide intuitive explanations of complex log entries in Logs Explorer for easier root-cause analysis and provide suggestions for how to fix issues that surfaced in an error reporting tool.
Duet AI is also incorporated into the Google BigQuery service to help developers write SQL and Python code to access and analyze data. It can generate full functions and code blocks, auto-suggest code completions and explain your code and SQL queries. Via a single SQL statement, a query can connect tables with the foundational AI models Google makes available via its Vertex AI service.
At the same time, IT teams can optimize prompts using BigQuery Studio tools, perform text analysis or generate new attributes to enrich a BigQuery data model. Duet AI also enables IT teams to generate vector embeddings in BigQuery to create semantic searches and recommendation queries.
It’s not clear whether generative AI capabilities will automate entire DevOps and DataOps workflows just yet, but many of the tasks that once required automation scripts will soon be automated by AI models embedded within DevOps platforms. The challenge and the opportunity is to determine how to make those capabilities available to drive DevOps workflows at unprecedented scale.
-
VMware Previews Spring AI to Accelerate Java Development
At a SpringOne at VMware Explore event this week, VMware demonstrated a tool that uses abstractions to make it simpler to build Java applications that invoke multiple artificial intelligence (AI) components with minimal changes to code.
The goal for the open source Spring AI project is to, for example, make it simpler to swap out AI models using an AiClient interface for invoking OpenAI and Azure OpenAI services. It also adds capabilities that promise to make it simpler for developers to query documentation using generative AI enabled by large language models (LLMs). An additional goal is to extend those capabilities to other Spring projects, including Spring Integration, Spring Batch and Spring Data.
In addition, the Spring command line interface (CLI) has been extended to support Spring AI and the Spring Boot framework has been extended to help set up dependencies and classes more easily.
The AI capabilities are part of several additions made to the widely used framework for building Java applications that address everything from automatically turning off instances of Spring running in a container, also known as scaling to zero, to using virtual threads to call external services in a way that reduces memory consumption.
Those additional capabilities promise to make it more efficient to run Java applications in, for example, a cloud computing environment.
Betty Junod, vice president of product marketing for modern applications and the Management Business Group at VMware, said generative AI capabilities should serve to make Spring more accessible by not having to click around as much to gain answers to questions.
In general, VMware is betting that a larger percentage of the Java applications built using Spring will be deployed on the Tanzu platform the company has built on top of Kubernetes. That platform streamlines the application development process in a way that reduces the cognitive load on developers by eliminating the need to know how to work with YAML files or use Docker Compose to create containers.
Despite the proliferation of alternatives to Java, it remains the primary programming language used by developers to build applications. The challenges involved with learning additional programming languages are high, so most developers would prefer to continue to use Java to build both monolithic and emerging cloud-native applications.
In the meantime, AI is already widely employed to help developers write code faster. AI isn’t going to replace the need for developers, but it will increase the pace at which applications are being developed. As such, DevOps teams will soon see a significant increase in the amount of code moving through DevOps pipelines. The hope is that AI will be applied in a way that enables those DevOps teams to manage increasingly larger codebases.
In fact, the way applications are built and deployed will drastically change now that the AI genie is out of the proverbial bottle. But it remains to be seen how quickly those advances will change the way DevOps is managed at a level of unprecedented scale.
-
New Relic Extends AI to Identify Alert Coverage Gaps
New Relic this week added additional artificial intelligence (AI) capabilities to its observability platform that make it possible to detect and resolve alert coverage gaps.
Camden Swita, a senior product manager for New Relic, said it’s not uncommon for DevOps teams to discover an issue that the existing alerts they put in place were not designed to address. New Relic is adding machine learning algorithms at no additional cost that are designed to identify anomalies indicative of those potential issues. The platform then surfaces recommended alerts that DevOps teams can apply or further customize as they see fit, he added.
Those machine learning algorithms ensure DevOps teams are tracking the right thresholds to ensure there are no blind spots that could adversely affect application performance, noted Swita.
New Relic is employing both predictive and generative AI to enable DevOps team to detect and resolve issues faster. The company has been using machine learning algorithms for predictive AI for several years and, more recently, started providing early access to a generative AI tool dubbed Grok.
The overall goal is to make the capabilities of an observability platform more accessible, Swita said. This is accomplished by surfacing issues without requiring a software engineer to craft intricate queries to interrogate metrics, logs and traces, he noted.
The AI capabilities also serve to reduce the amount of alert noise; it’s now possible to more easily understand the root cause of an issue and reduce the total number of alerts that might otherwise be sent, he added.
While there’s no doubt AI will help democratize DevOps by making it possible to write scripts and better manage workflows, it might be a while before DevOps teams trust the recommendations made. AI models typically require some time before they understand how an IT environment functions, and not every initial recommendation may be valid. In time, however, DevOps teams will find themselves increasingly supervising processes that AI models are executing.
In addition, there are competing approaches to building large language models (LLMs) that drive generative AI. For example, it’s not clear whether chaining LLMs together will be more efficient than creating a single large ‘parent’ LLM that has ‘child’ relationships with smaller LLMs, noted Swita.
In the meantime, DevOps teams would be well-advised to list tasks that are likely to be automated by AI models. Many of them will tend to be manual tasks that most software engineers don’t especially relish, so in that regard, there’s naturally a lot of interest in AI. It remains to be seen, however, whether various forms of AI might one day automate an entire DevOps workflow. In fact, most DevOps engineers are going to set a high bar for AI models that, at the very least, will be expected to perform a task as well as humans before it’s trusted completely.
-
VMware Preps Generative AI Assistants to Automate IT Workflows
At its VMware Explore 2023 conference today, VMware previewed what will become a suite of intelligent assistants that use generative artificial intelligence (AI) to automate the management of a wide range of IT tasks.
Krish Prasad, senior vice president and general manager for cloud infrastructure at VMware, said VMware will eventually build out assistants for managing individual platforms such as VMware vSphere and the integrated VMware Cloud Foundation (VCF) platform through which the company aggregates those offerings into a single platform.
VMware is creating those assistants, also known as co-pilots, using multiple large language models (LLMs) that the company uses to train AI models using code written by site reliability engineers (SREs), said Prasad. That approach ensures that the recommendations being made along with the code generated are of the highest quality, he added.
That approach will enable IT teams to identify configuration drift that might be adversely impacting performance or that could create a cybersecurity risk, for example, noted Prasad. The first of those assistants is being made available in beta for VMware Tanzu, a platform based on Kubernetes. VMware is also building an assistant for its NSX virtual networking software that analyzes security events to generate policies and rules that can be applied to firewalls in real-time as threats are identified.
It will then be up to each IT team to decide whether to implement those recommendations. Over time, it’s clear that the management of IT infrastructure is going to become increasingly automated as confidence in those recommendations increases, he added. In fact, the ultimate goal is to fulfill the promise of ‘lights-out’ IT management, said Prasad.
In effect, advances such as these will lead to the democratization of DevOps workflows as it becomes simpler to manage IT infrastructure at scale. IT organizations will be able to invest more resources in building and deploying applications rather than hiring specialists to manage IT infrastructure, noted Prasad.
In fact, the pace at which applications are built and deployed should dramatically increase in the months ahead, Prasad predicted.
The rise of generative AI creates a unique opportunity for IT teams to become more relevant than ever to their organizations. It will soon be possible to build and deploy a wide range of distributed applications capable of processing and analyzing data at the point where it is created and consumed.
While generative artificial intelligence (AI) has captured the popular imagination, it’s becoming apparent that multiple forms of AI will soon be used together to automate DevOps workflows at levels of unprecedented scale.
Machine learning algorithms, of course, are already used by many organizations to automate everything from analyzing log data to identifying security issues to surfacing patterns in tests made using multiple regression, function and unit testing tools. With the rise of generative AI, it is now possible to automatically generate code and create summaries of reports that multiple stakeholders can more easily understand. Generative AI doesn’t replace the need for previous use cases involving machine learning algorithms. Instead, generative AI makes use of a form of deep learning algorithms to create different types of AI models.
As a result, DevOps workflows will soon be invoking multiple classes of AI models to automate workflows in ways that should enable IT teams to automate processes at unprecedented levels of scale.
-
Low-Code and AI: Friends or Foes?
The lines are becoming blurred between low-code and new artificial intelligence (AI). Historically, low-code/no-code platforms have introduced software development automation through graphical user interfaces, enabling professional and citizen developers alike to quickly build workflows and generate applications. But the rise of large language models (LLMs) represents an existential threat to the status quo of low-code development.
So, are generative AI and low-code friends or foes? Since new generative AI can produce complex code in milliseconds given natural language prompts, the barrier to software development has never been lower. On the other hand, enterprises already have ingrained low-code practices and operations, which may stick around for a while. With that in mind, there are plenty of ways AI and low-code can coexist.
Below, we’ll consider if AI and low-code are diametrically opposed tools or complementary forces. We’ll see how AI and low-code are merging and consider the benefits of using them in tandem.
Unleashing the Power of AI
LLMs, like OpenAI’s GPT-3.5 and GPT-4, can deliver impressive software generation capabilities, such as creating code snippets and even complete programs given natural language prompts. Because of these remarkable capabilities, it makes sense that low-code platforms should be a bit wary of AI.
Artificial intelligence can actually deliver on some of the original promises of low-code, like software democratization, enhanced agility and filling the widening skills gap. “ChatGPT could allow users to make more significant changes to applications, potentially enabling organizations to achieve far more than with low-code and in half the time,” wrote Romy Hughes on devopsy.kinsta.cloud.
There are also plenty of ways LLMs could accelerate the DevOps side of the cloud-native equation. In addition to code generation, AI could help generate configuration manifests, run code tests and perform cybersecurity analysis. While AI can’t fully replace the nuance of human oversight, it can indeed automate more and more elements of the software development life cycle.
Congruence Between Low-Code and AI
Although it appears likely that AI will replace low-code, there are actually many opportunities for symbiosis between the two concepts. Rather than eradicate low-code platforms entirely, LLMs will likely become more embedded within them. We’ve already seen this occur as low-code providers like Mendix and OutSystems integrated ChatGPT connectors. Microsoft has also embedded ChatGPT into its Power Platform as well as integrated GPT-driven Copilots into various developer environments.
“Low-code and AI on their own are powerful tools to increase enterprise efficiency and productivity,” said Dinesh Varadharajan, the chief product officer at Kissflow. “But there is potential for the combination of both to unlock game-changing automation for almost every industry. The power comes from the congruence between low-code/no-code and AI.”
There is also the opportunity to train bespoke LLMs on the inner workings of specific software development platforms, which could generate fully-built templates upon natural language prompts. This would make the LLM more custom-built and relevant to the platform at hand and ease development fluidity. In addition to integrating existing LLMs, another potential synergy is that low-code solutions could help automate the training of new custom AI models.
Tips to Keep In Mind
As with any technology solution, the choices should be driven by concrete business outcomes. Start by defining the business objectives and let that determine the technological solutions to arrive at the outcomes you desire. In the context of low-code and AI, this might equate to deciding between which models to support.
Next, enterprises must avoid risk when using low-code platforms. And on the AI side of the coin, platforms will have to navigate the data privacy and legal complications of using public LLMs. Like any new technology, AI and low-code will require governance, said Varadharajan. “Organizations should step in to standardize the tools, processes and DevOps cycles. Only certain AI-powered tools should be approved for development where there are agreeable security, privacy and IP protection agreements in place.”
He also explained that a focus on change management would be paramount in reaping the benefits of both strategies. “I suspect that the greatest barrier to adoption and implementation will be change management,” he said. “Enterprise users are accustomed to certain ways of working and can be resistant to change, making them unlikely to embrace AI-powered low-code. Leaders should anticipate this and incorporate change management into their AI/low-code adoption strategy.”
Closing The Semantic Gap
Low-code came on the scene to help citizen developers create applications without requiring advanced technical experience. Yet, low-code still requires knowledge of the development platform as well as specific programming terminologies and formulas. But now, since generative AI behaves in response to natural language, it has the potential to instantly bypass this barrier.
“Generative AI closes this semantic gap,” said Varadharajan. “Generative AI serves as a translator, and users no longer have to understand exactly how the system works.”
As more AI pair programmers come onto the market, expectations are rising within all areas of software development, including low-code platforms, to integrate these advanced models. If leaders can navigate the technical change effectively, the outcomes will likely significantly increase agility and output.
“Ultimately, all developers will end up saving time delegating repetitive tasks to these AI-powered pair programmers,” said Varadharajan.
-
Teleport Assist Leverages Generative AI to Democratize DevOps
Teleport has made available a chat interface, dubbed Teleport Assist, that makes use of OpenAI’s generative artificial intelligence (AI) application programming interfaces (APIs) to enable DevOps teams to troubleshoot infrastructure issues more easily.
Using the natural language interface (NLI) provided by Teleport Assist, DevOps engineers can ask questions and give instructions in English to massive fleets of servers. Capabilities include being able to suggest commands and scripts for debugging and resolving common issues and then generating commands for review before they are applied.
In future versions of Teleport Assist, support for other cloud resources, such as SQL databases and Kubernetes clusters, will be added. Teleport Assist currently supports Secure Shell (SSH) and bash scripts, with support for additional languages and interfaces to be added over time.
Teleport CEO Ev Kontsevoy said it’s already clear that many of the scripts that DevOps engineers rely on today for IT management will be replaced by generative AI platforms. The average IT administrator will be able to invoke DevOps platforms via an NLI that eliminates the need for extensive programming expertise, he added.
In effect, the NLI surfaced by generative AI platforms will serve to democratize DevOps expertise, Kontsevoy noted.
It’s still early days as far as the adoption of generative AI by DevOps engineers is concerned, but it’s already apparent that its impact will be profound. It will increase the pace at which applications are built while making it easier for software engineers to manage large codebases. Most of the generative AI advances thus far improve developer productivity, but tools such as Teleport Assist show how generative AI will also make it easier for more organizations to adopt DevOps best practices that previously were limited to organizations that could afford to hire and retain significant numbers of software engineers.
At their core, generative AI platforms are fundamentally changing the way humans interact with machines. Instead of requiring a developer or engineer to create a level of abstraction to communicate with a machine, it’s now possible for machines to understand the language humans use to communicate with each other. A a natural language interface will soon enable DevOps engineers to ask generative AI platforms to not only surface suggestions but also debug applications and even write code. The accuracy and quality of those results will vary depending on how the large language models (LLMs) that trained those platforms were, but over time there will be a wide range of LLMs that are trained to be used within specific domains such as DevOps.
The issue is determining how to organize DevOps teams as more tasks become automated. Generative AI doesn’t replace the need for IT professionals, but it does make it simpler to manage IT at unprecedented levels of scale. In fact, at this point, the proverbial generative AI genie is out of the bottle, so there is no going back. The only thing left to decide is how quickly generative AI becomes pervasively applied.
-
Datadog Adds Additional Generative AI Capabilities to Platform
At its DASH 2023 conference, Datadog today launched Bits, a digital assistant that uses generative artificial intelligence (AI) from OpenAI’s ChatGPT platform to provide real-time recommendations to resolve issues surfaced by the company’s observability platform.
At the same time, the company announced it is extending the reach of that platform to enable IT organizations to monitor the large language models (LLMs) many are now embedding within custom applications.
Datadog also released an LLM Observability platform in beta, which brings together data from applications, models and various integrations to help engineers discover and resolve AI issues in applications such as performance degradations, drift and hallucinations in real-time.
Those extensions complement an existing ability to monitor the usage patterns of application programming interfaces (APIs) making calls to the OpenAI ChatGPT platform.
Yrieix Garnier, vice president of product at Datadog, said the generative AI capabilities in beta are enabled by data the company used to extend the core capabilities of OpenAI’s ChatGPT platform.
Bits AI learns from both customer data and other sources to answer queries via a natural language interface that makes the Datadog platform more accessible to a much wider range of IT professionals, noted Garnier.
The overall goal is to reduce the mean-time-to-resolution for issues by, for example, explaining errors and suggesting code fixes that can be easily shared with developers via a Slack communication channel.
Those capabilities extend the functionality of the machine learning algorithms that Datadog has already embedded within its platform, noted Garnier.
While the generative AI capabilities added to the core Datadog platform will make DevOps teams more efficient, it’s not quite clear what role those teams will play in monitoring LLMs created by data science teams. In theory, DevOps teams should be responsible for optimizing the performance of any element of an application, but most DevOps engineers today don’t tend to have much experience with LLMs that are created by data science teams.
At the very least, however, DevOps engineers should be able to share the root cause of an issue that is adversely affecting those LLMs with those data science teams.
It’s not clear how quickly LLMs will be incorporated into applications, but most DevOps teams should expect them to soon be pervasive. In effect, Datadog is now providing DevOps teams with the AI tools they will need to effectively monitor applications infused with AI capabilities. The issue is not so much whether DevOps teams will be taking advantage of AI to better manage application environments but how quickly. Given the overall level of complexity DevOps teams regularly encounter, few are likely to want to take on that challenge without some level of AI augmentation.
The challenge is determining exactly what DevOps tasks might be best handled by a machine that enables software engineers to focus on more complex issues they previously didn’t have the time to address.
-
Agile Planning With Generative AI
Generative AI will eventually impact the entire DevOps life cycle from plan to operate. I started as a developer but have been a product manager for most of my career; for me, the ‘Holy Grail of DevOps’ would be one where product managers (PMs) and business analysts (BAs) were able to define a future state of a business process and press a button to deliver it without any developers, designers or testers involved.
This dream is not practical in the near term and is not really desirable in the long term, either. PMs and BAs are good at understanding the needs of users and translating them into features but aren’t interaction designers. AI may be able to generate code and tests, but the software architecture for both will still require a deep understanding of the system to structure the solution properly, especially in enterprise SaaS development. So my dream is to build a team where the BAs can define the changes and a small team of very talented architects and interaction designers can realize those changes in 10% of the time it takes today without requiring a large team to implement the details. This is similar to what has happened in manufacturing where robots and numerically controlled machines are able to do the heavy lifting with the help of operators.
Planning is Key
When generative AI is actually building the solution, proper planning will be the entire key to success. A well-documented, well-thought-out plan will enable the AI to finish most of the heavy lifting. So the majority of the focus for the dream team should be to get the plan right and to document it at just the right level of detail. Just like I was taught by my first Agile coach 20 years ago, when it comes to documentation, you need just enough just in time.
In enterprise SaaS platforms like Salesforce.com and ServiceNow, the business starts with a newly implemented system in some initial implementation and then incrementally customizes this software to better meet their needs over time. So, instead of thinking about building a system from scratch, we must think of digital transformation as the continuous improvement of an existing system over time. That means the AI must start with a good model of the
● Existing business processes,
● Metadata configuration of the system and
● Dependencies between the settings.Development in the New World
What does this mean in practice? Let’s look at a day in the life of the dream team.
The BA will start by telling the AI what changes are desired at a high level. The AI will help refine the requirements by filling in missing details. This will happen through an iterative process of expanding, refining and summarizing until the requirements are clearly defined. Just as lint tools check source code today, the AI will examine the requirements to ensure the user stories are complete, with acceptance criteria well defined. During this process, the AI will do a detailed dependency and impact analysis to provide a heads-up to the BA when there are any unwanted side effects.
This refinement session will be more of a conversation between the BA and the AI than a one-sided writing session. After several iterations, a complete set of user stories will be created with all of the necessary details spelled out.
Once the requirements are complete, the BA will be joined by a user interaction designer and software architect.
The interaction designer will define the interaction spec through a combination of natural language description of the new processes and a functional prototype. Both of these elements will be created and refined through interactions with the generative AI. Prototypes will no longer be simple graphic artifacts to be coded by front-end engineers. The AI will generate a working prototype in the framework of your choice on demand through a combination of verbal requests and traditional point-and-click.
They will review the plan and begin designing the solution’s architecture which will result in a natural language software spec that documents the data model changes and high-level specifications for the classes that make up the solution. It is likely that the data model changes will be created in a dev environment on demand, with the ERD updated to reflect the new model.
Depending on the type of changes, the architect and interaction designer may collaborate and iterate on the specs until the “how” is well defined. When the design is ready, the dream team will meet to review the design and sign off on the approach.
Generate and Iterate
Once the plan is approved, it’s time for the AI to generate a first-pass solution. We hope and expect that generating the solution will be something that happens in minutes. In this scenario, the dream team will take a coffee break and then come back to the meeting room to review the working solution.
There is nothing like playing with the actual solution to remind you of the edge cases you forgot or to realize how tedious it is to actually use a new feature. But that’s okay because the team can quickly update the requirements to account for these changes and push the generate button again.
In fact, they can tinker to their heart’s content. But like all great artists, they must decide when it is good enough to ship.
In the near term, we will not be able to blindly rely on generative AI, so the architects will have to do code reviews, just as they do today with junior developers. Likewise, the AI may not understand all of the possible dependencies in a system. So there will be a transition period where the process is semi-automatic.
Eventually, the notion of a two-week sprint will seem anachronistic and a daily cycle of define and release to production will be commonplace. Members of the dream team will spend more time worrying about the high-level usability and effectiveness of their designs and less time struggling with actual implementation, which should make their jobs much more rewarding and the work product much better. I am looking forward to it.
-
Digital.ai Embraces Multiple AI Classes to Advance DevSecOps
Digital.ai has committed to adding generative artificial intelligence (AI) and additional machine learning capabilities to its namesake platform for managing DevSecOps workflows.
The company plans to make available in alpha a generative AI tool to create test cases and user stories via large language model (LLMs) access via a natural language interface. In addition, generative AI will also be used to identify high-risk changes to code and improve the overall governance of DevSecOps workflows.
At the same time, Digital.ai has extended the machine learning algorithms it already provides to provide predictions and scoring. Additional capabilities include flow acceleration recommendations, change risk predictions and service management process optimizations.
Digital.ai CEO Derek Holt said as AI continues to evolve, the company will use a mix of public and private LLM platforms to ingest AI into DevSecOps workflows. In effect, the future of AI will almost be multimodal as different classes of AI technologies are used to automate workflows, he added.
The goal is to safely break down the silos of data that today make it difficult to cost-effectively build and deploy secure software without compromising the rate at which it is developed, noted Holt.
It’s still early days as far as the infusion of AI in workflows is concerned, but it’s apparent that many of the manual tasks that prevent organizations from fully embracing best DevSecOps practices will be automated.
At the same time, increased reliance on digital processes, coupled with more stringent pending regulations, are reducing the overall tolerance organizations have for insecure software. The only way to address that rising sense of application security urgency—at a time when more complex cloud-native applications are being built and deployed—is to rely more on automation. In fact, the race is now on to secure software supply chains before regulations that will have stiff penalties for security lapses go into effect.
Most of the AI advances in software development that have been made thus far benefit developers. However, many of those same core AI advances will soon be applied to software engineering. Many of the DevSecOps bottlenecks that exist today will start to fade away as various classes of algorithms are able to both identify and make recommendations to eliminate them. As those capabilities evolve, it should also serve to help reduce the long-standing cultural divide between application development teams and cybersecurity professionals that is at the root of the software supply chain issues that organizations are still struggling to resolve.
In fact, many organizations would be well-advised to review their current workflows with an eye toward determining which processes can be automated. The days when AI was sparingly used are now giving way to a new era of software development that promises to dramatically accelerate the rate at which applications can be built, deployed and updated. The challenge, as always, is finding a way to incorporate these advances into workflows that were not created with AI in mind.