Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: PyPI complies with a “string of subpoenas,” and LTO continues to grow, despite predictions of its demise. (more…)
Red Hat today unveiled a technical preview of Ansible Lightspeed with IBM Watson Code Assistant, a generative artificial intelligence (AI) service specifically trained on domain-specific data collected via the Ansible automation platform.
At the same time, Red Hat is also making available an event-driven capability within version 2.4 of the Ansible platform that enables runbooks and other tasks to automatically execute when received from a third-party tool or application.
Both capabilities were announced at the Red Hat Summit and are part of an ongoing Red Hat effort to democratize IT automation.
First previewed within the Project Wisdom initiative launched by Red Hat and IBM Research last year, Ansible Lightspeed IBM Watson Code Assistant is based on a large language model (LLM) that was developed using Ansible data. Watson Code Assistant, currently in technical preview, is a service that automatically generates suggestions to improve code as developers create an application. IBM promised to make IBM Watson Code Assistant available later this year.
Red Hat is now applying that generative AI capability to the YAML code that DevOps teams need to create Ansible playbooks. IT teams can create a simple natural language prompt that will translate their domain expertise into YAML code for creating or editing Ansible Playbooks. Red Hat and IBM, however, will not automatically use the code from an organization to fine-tune their AI model without express consent.
Rich Henshall, director of product management for Red Hat Ansible, said as IT environments become more complex, it’s clear many are hitting a ceiling in terms of what can be automated using legacy custom scripts. Ansible Lightspeed IBM Watson Code Assistant, along with an event-driven capability, will make it much simpler to automate the management of IT environments at scale using a declarative framework, he added.
Ansible, for example, can now receive events from third-party tools, determine next steps and consistently initiate an automated response using Ansible playbooks. Ansible can also consume events from third-party monitoring, observability and IT analytics tools from providers such as Cisco ThousandEyes, CyberArk, Dynatrace, F5, IBM Instana, IBM Turbonomic, Palo Alto Networks and Zabbix with more to come, noted Henshall. Red Hat is also making available supplementary content for Red Hat OpenShift, Red Hat Insights, Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform and ServiceNow, in addition to enabling IT teams to create their own custom integrations.
At this juncture, it’s not clear why more IT processes have not been automated using Ansible or other frameworks. Many DevOps teams continue to rely on custom scripts that typically don’t scale and are generally poorly documented. Any time the individual who wrote those scripts leaves an organization, it’s incredibly difficult—if not impossible—for the original scripts to be maintained, noted Henshall.
One way or another, it’s clear that a more scalable approach to IT automation is required, especially with the rise of more complex cloud-native application environments. It’s no longer economically viable for most organizations to rely on legacy processes that require a small army of IT professionals to maintain.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Sources say it’s an open source future for OpenAI, and improving AI perf with Mojo. (more…)
Boomi announced today it is applying generative artificial intelligence (AI) to enable organizations to automate integrations via a natural language interface.
Boomi CEO Steve Lucas said Boomi AI makes it much simpler for professional and citizen developers alike to create integrations at scale using generative AI platforms to invoke the low-code engine that Boomi already provides to create integrations between thousands of applications.
Those generative AI capabilities will complement a wide range of AI capabilities based on machine learning algorithms that are already embedded in the Boomi platform, he added.
Generative AI’s Impact on Integration and Automation – Steve Lucas, Boomi from Techstrong.TV on Vimeo.
Boomi has been widely used to drive business process integration via an integration platform-as-a-service (iPaaS) environment it delivers as a managed service. The low-code platform the company created significantly reduces the need to rely on professional developers to write procedural code using Java, for example, to create those integrations.
Generative AI capabilities make it possible for almost anyone of any skill level to create those integrations across multiple types of application programming interfaces (APIs), noted Lucas.
That capability also makes it possible for Boomi to notify end users when, for example, a data attribute in an application has changed and ask them if they would like to apply that change across their entire application environment.
In addition, Boomi AI can also interpret the intent of a prompt to better orchestrate processes in addition to providing an audit history in natural language that explains how decisions were made and what outcomes occurred as a result. As part of that effort, Boomi algorithms are also trained to avoid biases and unfairness with established ethical guidelines.
Boomi’s AI engine is trained using anonymized data collected from approximately 20,000 customers that have created more than 200 million integrations using the Boomi platform. Those integrations have been exposed to third-party large language models (LLMs) that the company is using to train Boomi AI, said Lucas. Boomi is not storing any customer data to train its AI models.
DevOps teams, of course, spend hours creating integrations that should be much simpler to create, test and validate with the rise of generative AI. In fact, the number of integrations being created to drive a wide range of digital business transformation initiatives should substantially increase.
In the meantime, DevOps teams would be well-advised to start evaluating which tasks they manually perform today that could instead be automated by a generative AI platform. Many of the routine integrations that DevOps teams create today using low-level APIs will be increasingly automated by end users. The ultimate goal should be to identify more complex tasks that previously DevOps teams would not have attempted simply because there wasn’t enough time.
It’s still too early to determine what impact generative AI will have on the way IT is managed, but DevOps teams that are typically committed to automating as many processes as possible should be at the forefront of adoption. The challenge is going to be simply keeping track of the rate of change that might soon be enabled by these platforms.
SapientAI emerged from stealth today to automate the writing of test code using a combination of generative artificial intelligence (AI), machine learning algorithms and data science.
Fresh from raising $5 million in seed funding, SapientAI CEO Rishi Singh said the goal is to leverage AI to push application testing further left using an AI Test Coder the company built.
The concept of AI Test Coder is similar to the co-pilot capabilities that GitHub and Google are providing to help developers write better code faster, but also goes beyond generative AI to add deeper analysis and contextual understanding to the tests it automatically generates, noted Singh.
In the longer term, the ability to use AI to create tests will result in the deployment of more secure, higher-quality applications because there will be no need to short-shrift application testing processes in the interest of meeting deadlines, added Singh.
Prior to the rise of AI, building application tests has historically been prone to error. AI Test Coder is designed to be a plug-in for an integrated development environment that will leverage the large language models (LLMs) created by OpenAI and cloud service providers to employ natural language to create tests based on an analysis of the application environment created using a combination of machine learning algorithms and data science techniques, said Singh.
As part of the effort, AI Test Coder instantly inspects all software code to understand the software testing environment, including any incorrect assumptions that stem from errors and bugs, before generating any test code. It then leverages code intelligence to understand existing code and automatically identify and detect patterns to anticipate any new software issue that may arise. AI Test Coder also locates where code changes will have an effect before any test code is created. This eliminates time that would otherwise be wasted on manually debugging code.
Once those steps are completed, Test Coder then generates more accurate tests without relying on the current trial-and-error approach to testing that can slow application development, said Singh.
In effect, AI Test Coder provides each developer with their own assistant dedicated to managing testing tasks that ultimately improve quality assurance, including removing many of the vulnerabilities that create application security issues that have become the bane of software supply chains.
It’s too early to determine precisely what impact AI will have on application development and deployment, but many of the manual tasks that create a sense of drudgery are about to be eliminated. That means the overall pace at which applications can be built and deployed is about to be greatly accelerated. DevOps teams are likely to find themselves pushing more code simultaneously through pipelines than ever.
Naturally, AI will soon be applied to not just writing code and tests faster, but also DevOps workflows. The issue now is determining which of the tasks that today require manual intervention will be eliminated in a way that enables DevOps teams to focus more of their time and effort on more complex challenges.
At its SLOconf event today, Nobl9 added a Query Tester tool that validates whether queries made against the data its service level objectives (SLO) management platform collects will deliver meaningful results. Nobl9’s SLO management platform collects data in real-time from data sources such as New Relic, Datadog and Dynatrace to help customers meet SLOs.
In addition, a Query Checker tool will now automatically put any new SLO into a testing state to automatically validate that the query against that SLO will run correctly.
The company has also improved the precision of its calculations to provide DevOps teams with more accurate, reliable and actionable insights into the performance of services and added a Metrics Health Notifier toolset that provides insights into anomalous data being collected from various sources as well as what data might be missing. Nobl9 has also added a Custom Query Delay tool that gives DevOps teams more control over when data is pulled along with access to data source logs.
Kit Merker, chief growth officer for Nobl9, said the overall goal is to improve the overall resiliency of its platform for managing SLOs.
Finally, Nobl9 also announced that its platform is now available for the Google Cloud Platform (GCP) in addition to existing support for Amazon Web Services (AWS). As part of that GCP support, Nobl9 made available an educational site that provides example of how DevOps team can take advantage of the large language models (LLMs) that Google is making to improve SLOs or generate an SLO that is automatically converted into a set of YAML files.
A survey of 314 IT professionals conducted by Dimensional Insight on behalf of Nobl9 found 69% have implemented SLOs, with more than three-quarters of those respondents crediting SLOs for preventing business disruptions. A total of 80% also noted their organization has an increased focus on system reliability, with just over a quarter (27%) saying SLOs saved their organizations a half million dollars or more. A full 95% said SLOs are also driving better business decisions.
However, 97% also reported it is difficult to manage SLOs because of a lack of reporting, lack of application and environment support and interoperability issues.
Nobl9 is trying to spur greater adoption of SLOs by making available an open source SLO specification that defines a common interface for constructing SLOs across a Git-based workflow. Nobl9’s platform based on that specification has now been integrated with 24 data sources, said Merker.
SLOs, of course, are not a new idea. They have been employed as a metric to track the performance of IT services for decades. However, as more microservices-based applications are built and deployed, it’s becoming more challenging to maintain SLOs across applications that have many more dependencies than legacy monolithic applications.
Ultimately, each IT team needs to provide some sort of objective benchmark that assesses their overall effectiveness at delivering application services. SLOs-as-code are intended to make it simpler to gather the metrics that confirm whether service levels are being achieved. Those SLOs are then tracked and integrated across any set of application services a DevOps team cares to track.
Google is previewing an instance of a generative artificial intelligence (AI) service that surfaces code recommendations to make it possible to build high-quality applications faster.
Specifically, Duet AI for Google Cloud surfaces code recommendations in real-time, generates full functions and code blocks and suggests fixes for vulnerabilities and errors in the code that it discovers. It also promises to enable DevOps teams to create custom large language models (LLM) based on their own code. In fact, GitLab and Tabnine are, respectively, already integrating LLMs running on Google Cloud with their continuous integration/continuous delivery (CI/CD) platform and testing tools.
Chen Goldberg, vice president of engineering for Google, told attendees at the Google I/O 2023 conference that Duet AI for Google Cloud will fundamentally change how developers will be able to build applications, regardless of their skill level.
Duet AI for Google Cloud was trained using data collected from Google Cloud using sample code and documentation to fine-tune behaviors and patterns.
Duet AI for Google Cloud is available under an AI Trusted Tester Program and provides developers with many of the same capabilities that can be found in the GitHub Copilot tool that is based on the LLMs created by OpenAI.
In addition to making Duet AI for Google Cloud accessible via both its tools and managed services, Google said it also plans to integrate the service with multiple integrated development environments (IDEs).
It’s still early days as far as the use of generative AI within DevOps workflows is concerned, but it’s clear that the amount of code being created by developers is about to increase exponentially. Hopefully, AI technologies will also one day help software engineers find ways to manage the massive volume of code moving across their DevOps pipelines.
Generative AI platforms will fundamentally change the way humans interact with machines. Instead of requiring a developer to create a level of abstraction to communicate with a machine, it’s now possible for machines to understand the language humans use to communicate with each other. Using a natural language interface, developers will soon be asking generative AI platforms to not only make suggestions but also debug applications and, eventually, write the code used to create an application.
At this point, like it or not, the generative AI genie is out of the proverbial bottle. Just about every job function imaginable will be impacted to varying degrees. In the case of DevOps teams, the ultimate impact should involve less drudgery and toil as many of the manual tasks that conspire to make managing DevOps workflows tedious are eliminated.
In the meantime, DevOps teams should spend some time identifying which tasks will become automated by generative AI and other technologies that employ machine and deep learning algorithms. It doesn’t make much sense to onboard new members to a DevOps team only to discover that the tasks assigned to that individual are about to be automated. Hopefully, when all these AI technologies are finally implemented, the entire DevOps experience will be better than ever.
New Relic has added a tool that takes advantage of generative artificial intelligence (AI) to make it simpler to query the telemetry data collected by its observability platform via a natural language interface (NLI).
Jemiah Sius, director of developer relations for New Relic, said New Relic Grok will make the New Relic One platform more accessible to a wider range of IT professionals, especially those that haven’t mastered New Relic’s proprietary programming language that enables DevOps teams to identify the root cause of an issue.
New Relic Grok automatically pinpoints code-level errors in integrated development environments (IDEs) in addition to analyzing code, stack traces and production telemetry to suggest fixes. New Relic is providing this capability via integration with the large language models (LLMs) created by OpenAI, which is providing the foundation for a broad range of generative AI capabilities through Microsoft. New Relic has a long-standing alliance with Microsoft, added Sius.
Using plain language, anyone can generate a system or app health report complete with anomalies, issues and recent deployments in a way anyone can understand without having to create and filter dashboards, he noted.
New Relic Grok also makes it simpler to identify instrumentation gaps that can then be addressed using the instructions it provides. It can also set up missing alerts and automate alerts using the open source Terraform infrastructure-as-code (IaC) tool.
Finally, IT teams can also use New Relic Grok to manage accounts, users and user access, data retention rules, usage, billing and other administrative tasks.
New Relic previously invested in machine learning algorithms to automate observability tasks. These AI extensions to the New Relic platform are part of a larger effort to extend the reach of the New Relic One observability platform further left toward developers and IT administrators rather than just site reliability engineers (SREs), said Sius.
However, New Relic doesn’t anticipate that generative AI will eliminate the need for SREs to use its programming tool to solve more complex problems, he added.
The overall goal is to provide development teams with frictionless access to observability data at every stage of the software development life cycle to reduce mean-time-to-detection (MTTD) and mean-time-to-resolution (MTTR) of issues.
In general, most organizations are still in the early stages of achieving full-stack observability. A recent New Relic survey found only 27% of respondents have achieved full-stack observability and only 5% claimed they have a mature observability practice in place. A third (33%) of respondents also said they still primarily detect outages manually or based on complaints, the survey found. On the plus side, the survey also found nearly three-quarters of respondents said C-suite executives in their organization are advocates of observability, and more than three-quarters of respondents (78%) saw observability as a key enabler for achieving core business goals. However, more than half (52%) of respondents said they experienced high-business-impact outages once per week or more and 29% said they take more than an hour to resolve those outages.
AI, of course, promises to improve reliability. At the very least, it could make it easier to identify issues long before a major disruption occurs.
‘Adapt or perish’ is the mantra in the tech world, and it holds particularly true if you’re a developer.
Because of how dynamically technology evolves, developers face greater pressure than most to be ahead of the curve in adapting and becoming proficient in the best tools becoming available as a result. ChatGPT is the latest such tool.
While there has been talk of ChatGPT being a “job killer” and horror stories are making the rounds about a dystopian future in which it renders half of humanity meaningless, the reality is much more nuanced. The market will evolve as it has with the advent of any technological advance, and those who can leverage such leaps to their advantage will continue to be effective.
For developers, ChatGPT represents a chance to introduce efficiency in their tasks, scale up productivity and minimize errors and deficiencies. Used wisely and appropriately, it can help developers become better at their jobs by honing their abilities and enabling them to work more creatively within their existing functions.
Here are just some of the few areas where ChatGPT can bring tremendous value to developers:
Writing code is a complex process. Debugging it is harder. And in between those two is the process of actually testing the code. ChatGPT allows developers to enable real-time or prompt-driven code testing at different instances to ensure the coding process becomes more efficient.
ChatGPT provides a wide array of suggestions and assertions based on the functions and variables used in that particular code to allow for wholly tailored testing of the code for that particular environment. Doing so can be particularly helpful in the subsequent debugging and documentation phases.
Ask any developer what the worst part of their job is, and debugging is likely to be relatively high on their list. As an old saying goes, “An average developer spends hours writing code and days debugging it.”
However, with ChatGPT, that may soon become a thing of the past; it can be leveraged to debug thousands of lines of code for error identification and then debug again based on any error message received.
Regardless of whether you’re a seasoned veteran or just starting out, in this particular case, ChatGPT can be invaluable when scanning code and suggesting possible fixes. Used effectively, this can help developers perform their functions more efficiently by eliminating the laborious hours spent identifying the problem to fix.
Furthermore, this should prove particularly helpful when writing test cases and documenting the debugging process to identify what caused the bug in the first place.
As mentioned above, ChatGPT can be leveraged to assist in or, in some limited cases, perform the entire process of documentation.
It can analyze complex code, contextualize and understand its logic and develop an appropriate explanation of the code’s functionality. More importantly, the specific language used in such documentation can be adjusted to ensure it is tailored to the skill and comprehension level of the person reading it.
Imagine not having to explain and re-explain how the code works based on who reads the documentation. As a result, developers can spend more time coding rather than documenting what their code does.
The most exciting, if not the most effective, way ChatGPT promises to revolutionize what it means to be a developer is code generation. With its code generation functionality, ChatGPT effectively functions as a personal coding assistant that understands the users’ coding preferences and natural language and turns that into executable code.
Doing so promises to bring unparalleled efficiency to all code generation, as simple command prompts can be used to produce code. In other words, code generation that would have taken minutes can now be done within seconds.
Moreover, this allays the fears of complete code generation by ChatGPT, as programmers can assume an oversight role over its generated code. This, coupled with some of the other benefits discussed, has a lot of time-saving potential and it can maximize productivity.
Machine learning is yet another exciting aspect that promises to drive the future of coding and development in general. ChatGPT can be used across the board to clean, transfer and appropriate particular datasets to be used for testing and training.
With the ability to deal with issues such as missing values and miscoded variables, ChatGPT can enhance the overall value that can be driven from any particular dataset while bringing more efficiency to the table.
This is where the expansive aspect of ChatGPT, as mentioned earlier, comes into play. ChatGPT is more than just a programming developer’s friend. ChatGPT is fully capable of being integrated into the Unity Editor. This is particularly significant as this allows for extensive 3D modeling and in-game designs.
Digital prompts can be converted into usable 3D images and models, allowing faster and more elaborate renders. This promises better productivity, with the scale of in-game designs set for unprecedented growth in terms of both details and content.
ChatGPT is already pushing the limits and boundaries in terms of generating and documenting code. Additionally, it addresses another vital issue related to coding and programming as a whole: Identifying potential vulnerabilities.
Using security tools such as a firewall or password manager is not enough protection. ChatGPT can be leveraged to generate code and peruse and review existing code to weed out any potential vulnerabilities. Not only can it point out potential vulnerabilities, but it also helps carry out parallel quality assurance to enhance the overall security and quality of that code.
Pair that with the conventional and reliable privacy and security elements, and you’ve generated near-impenetrable code for secure deployment.
Perhaps the most hyped-up benefit of ChatGPT is the proliferation of extensions available since its release. While the use of automation to oversee the completion of tedious and monotonous tasks is nothing new, ChatGPT allows for an unprecedented degree of customizable extensions. Developers can generate extensions based on their own usage patterns and preferences.
As is the case with any new technological wonder, ChatGPT is a resourceful tool, but its full potential can only be leveraged by those that understand it and know exactly where and how ChatGPT may prove most productive. However, the very fact that we do not yet understand the full extent of its capabilities should be cause for caution.
ChatGPT has already faced mounting criticism related to how it handles users’ data privacy, particularly that of minors.
It is critical for organizations to devote appropriate resources to fully assess exactly how ChatGPT may bring the most value to them without negatively impacting security and data privacy.
It’s only been a couple of months since OpenAI introduced its latest low-cost API for developers to leverage ChatGPT in their applications and already many engineering teams have jumped headlong into new integrations. But developers experimenting with the possibilities of how ChatGPT can boost their software should temper that with some risk analysis and threat modeling before deployment. So say a range of cybersecurity and risk experts, who warn organizations that leverage the ChatGPT API in their software that they’ll need to navigate a minefield of privacy, data governance, and security operational risks.
“I use ChatGPT Pro and think it’s an exciting tool for holiday cards, poems for my kid, writing inspiration, first drafts and search,” says Patrick Hall, data scientist and co-founder of BNH.AI, a boutique law firm focused on minimizing legal and technical risks of AI. He’s also the author of the upcoming book Machine Learning for High-Risk Applications. “I get concerned about generative AI used as a consumer product in high-risk domains, such as consumer finance or the law.”
With that framing in mind, the following are a few of the areas that experts like Hall suggest that developers, DevOps teams and CTOs start thinking about to avoid some potentially costly unintended consequences of ChatGPT integration in their software.
One of the most immediate and obvious risk concerns IT leadership, DevOps teams and privacy experts are worried about is what happens to sensitive data when users enter it into a ChatGPT-backed prompt—whether directly or through an app using the API. Be it a customer entering personally identifiable information (PII) into an external-facing app or an employee entering proprietary company information in an internal tool using ChatGPT’s API, these use patterns open up a world of data exposure and regulatory compliance concerns. The good news is that OpenAI does not use data submitted through the API to train its models, but this shouldn’t automatically give devs peace of mind. They’ll need to be sure that using generative AI in an app truly aligns with their company’s privacy policies and with any local, federal or international data privacy laws their firm is beholden to, said Hall.
“Although OpenAI appears to maintain a robust security and privacy program and there are steps you can take on your own to limit data sharing, some people are still worried,” said Walter Haydock, founder and CEO of StackAware.
Among Haydock’s side projects, he is currently developing a tool for sanitizing data before sending it to OpenAI. Called GPT-Guard, it’s still in beta and it would need to be run on a local machine or in the cloud to sanitize data before sending it via API. But it provides some rapid prototyping of the kind of methods and tools developers may need to think about as they design their applications to interact with ChatGPT.
Developers should also be thinking through intellectual property infringement concerns when it comes to the responses generated by an app tapping into ChatGPT, Hall said.
“Devs need to consider whether a generative AI system may be coughing up other people’s licensed or copyrighted intellectual property as responses,” he said.
This is likely going to be a tricky situation that will likely be tested by the courts in the coming years. For example, will applications using ChatGPT be required to compensate licensed copyright holders for certain kinds of responses using their material? These are the kinds of questions that developers and the organizations they work for should be asking themselves, said Frank Huerta, CEO of Curtail Security and a longtime DevOps expert.
“For example, what’s the mechanism for resolving IP and copyright disputes?” he asked. “Is ChatGPT going to have its own IP? Is it going to generate it—and will it put that in the public domain? Who owns the copyright to a generated response? None of that has been considered yet, and it’s going to come up fast.”
In addition to all of the concerns about the data, developers integrating ChatGPT into their applications also need to consider what that does to the software’s attack surface. ‘Never trust user input’ has always been a good rule of thumb for developers, and that advice is just as true for large language model (LLM) prompts as anything else.
A recent example scenario laid out by a data scientist on Twitter showed one of the myriad ways that malicious actors could carry out prompt injections in the same way they’d carry out a SQL injection. In that scenario, if a recruiter used ChatGPT to crawl LinkedIn profiles for a certain keyword and send an email to the people whose profiles included that keyword, it’d be possible to add text to a profile that sends an email chiding the recruiter for using an automated tool to send out emails. That’s a fairly low-stakes example, but it shows the possibility for injection attacks, said Andy Patel, a security researcher for WithSecure who has been digging into AI risks.
“So, if you are integrating it, you should think about things like that. You should think about the fact that people will determine that you are using a large language model to do something, to automate something, and then they’ll essentially SQL injection attack it,” he said.
Finally, developers and software design teams should be cognizant of the potential problems that AI bias can introduce into their software if they lean on the generated output from ChatGPT to trigger actions or to inform the user.
“Generative AI systems continue to make wrong, offensive or otherwise problematic content. Being wrong can give rise to negligence issues and biased content can lead to discrimination issues,” Hall warned. “Don’t embarrass yourself, offend others or run afoul of the law by not testing and further protect yourself by screening the heck out of generated content.”