devopsy.kinsta.cloud

Tag: generative AI

Diffblue Integrates Generative AI-Based Testing Platform With GitLab

Diffblue this week generally made available an integration between its automated unit testing platform for Java and the DevSecOps platform from GitLab.

The Diffblue Cover platform makes use of generative artificial intelligence (AI) to automatically create unit tests based on reinforcement learning technologies that don’t rely on LLMs—avoiding their drawbacks of sometimes introducing hallucinations and also requiring human review.

Diffblue CEO Mathew Lodge said the integration with the continuous integration capabilities found in the premium and ultimate editions of the GitLab platform would, for example, streamline regression testing in a way that should ultimately improve both application quality and security. Diffblue Cover, for example, can update tests as much as 250 times faster than a human developer can write them manually without developer review.

That approach also serves to reduce the level of friction many DevSecOps teams encounter when bottlenecks involving testing processes emerge, noted Lodge. The overall goal is to make it simpler for developers to test as much of their own code as possible before integrating it into a build, he added. Otherwise, developers will get fed up because testing is continuously breaking the build, noted Lodge.

Ultimately, instead of having to write unit tests, developers become supervisors of a platform that automatically generates them on their behalf, said Lodge. The job of a developer doesn’t go away, but it does fundamentally change, he added.

To achieve that goal, developers need to be able to access a platform that writes the tests and can then also execute them automatically. If it takes too long to create the test, chances are high that most developers won’t run it. On average, writing and evaluating a single unit test can take a developer 10 minutes. Over the course of any project, thousands of tests need to be written, so the amount of time testing takes away from coding is often much greater than most IT leaders fully appreciate.

Automating those tests should improve developer productivity as more time is available to focus on writing code rather than testing. That doesn’t necessarily eliminate the need for a dedicated testing team, but it does mean that more tests will be run without slowing down the overall pace of application development.

Developers naturally want to be able to test code at the very instant they create it. AI platforms can make that happen by, for example, employing reinforcement learning to write unit regression tests.

Most developers are not going to resist assuming more responsibility for testing if the tools to automate that task are more accessible. Instead of merely shifting responsibility for testing left toward developers, DevOps teams need to find ways to streamline the process. Otherwise, testing just becomes one in a series of tasks that are being shifted to developers in ways that many of them are coming to resent.

It may take some time before AI is fully integrated into software engineering but it’s clear with each passing day more previously manual tasks are being automated. Among the lowest hanging fruit for applying AI to software engineering are clearly testing processes that, if truth be told, few indeed enjoy conducting.

January 25, 2024
Will the Rise of Generative AI Increase Technical Debt?

In the last year and a half, we’ve seen a glut of new generative AI-related tools reach the market. Various AI pair programmers powered by large language models (LLMs) are becoming integrated into software development environments. The number of new generative AI startups is staggering, and generative AI seems to be the new “catch-all” to accelerate software development and operations.

In this new paradigm, AI tooling accumulation seems inevitable. Yet, the explosion of new AI APIs, libraries and frameworks has the very real potential to increase technical debt and, in the process, overwhelm DevOps teams. “These new applications and capabilities will require many new MLOps processes and tools that could overwhelm any existing, already overloaded DevOps team,” said Bob Quillin, vFunction chief ecosystem officer.

Below, we’ll consider if and how new generative AI tooling has the potential to exacerbate technical debt. Notably, the rush to adopt AI, shifts in the AI tooling landscape and accelerated code output could all be producers of technical debt. Choosing your tools correctly, iterating thoughtfully and upskilling teams will be important guidelines to mitigate negative outcomes. That, and creating unique solutions to ‘flip the coin’ — using AI to consolidate code and reduce existing technical debt.

How AI Will Contribute to Technical Debt

The rise of generative AI-related tools will likely increase technical debt, both due to the rush to hastily adopt new capabilities and the need to mold AI models to suit specific requirements. “New LLMs and generative AI applications will undoubtedly increase technical debt in the future, or at a minimum, greatly increase the need to manage that debt proactively,” said Quillin. “It starts with new requirements to continually manage, maintain, and nurture these models from a broad range of new KPIs from bias, concept drift, and shifting business, consumer, and environmental inputs and goals,” he said.

Incorporating AI may require a significant upfront commitment, leading to additional technical debt. “It won’t be just a build-and-maintain scenario, but rather, the first of many steps on a long road ahead,” said Prince Kohli, CTO of Automation Anywhere. Product companies with a generative AI focus must invest in creating a data and model strategy, a data architecture to work with AI, controls for the AI and more. “Technology disruptions and pivots such as this always lead to this kind of technical debt that must be continually paid down, but it’s the price of admittance,” he said.

AI is a tumultuous space, and some degree of technical debt will likely be produced by future organizational shifts. “The pace of change and innovation in the AI field itself is exceedingly rapid,” said Juan Orlandini, CTO of Insight North America, citing the staggering emergence of new AI startups and tools. “There will be consolidation, and also, unfortunately, a number of the startups will cease to exist.” When this occurs, teams must grapple with unsupported packages or tools.

Another type of debt is the nature of AI and how it accelerates development across the board. “One of the great values of these tools is the acceleration they bring to development,” said Orlandini. “Simply put, we can put out more features faster.” The other side of this double-edged sword is that AI accelerates the creation of all programming endeavors, even those that become technical debt.

Generative AI-related tools are most likely to accelerate the pace of change and deliver new code development outcomes more quickly. “In the process of consuming their output, there is a possibility of accumulating technical debt if the usage is broad, rapid, and pursued indiscriminately,” said Ajay Sabhlok, CIO and CDO, Rubrik. “Developers, in this case, may be too focused on program execution and forget to pay attention to code complexity, repetition or use of non-standard logic.”

Ways to Mitigate AI Technical Debt

Thankfully, there are specific methods organizations can adopt to manage new technical debt. For starters, you should focus on creating smaller proof of concepts and then iterating as you learn, said Kohli. “When leaders like the head of technology or IT are deploying these processes, they should first create small agile teams that learn fast and disseminate those learnings efficiently,” he said.

Also, technical debt is more likely to accrue when the wrong platform or tool is chosen. Therefore, “organizations should be very deliberate in choosing what products to leverage,” said Orlandini. He encourages IT leaders to avoid relying on companies or products with flashy presentations and poor tech or great tech with poor business acumen. “Our industry is rife with stories of great products with terrible business plans — which ultimately leads to their demise,” he said.

Other experts similarly foresee upcoming shifts that may affect internal operations. “Companies should keep their options open when considering AI model vendors or tool providers,” said Kohli. “They must keep their ‘hands on the wheel’ and be prepared for turns.”

Just like any new technological adoption, generative AI requires forethought. Therefore, putting the same amount of planning and preparation into deploying generative AI as we do for developing coding standards around manual coding could help minimize technical debt, said Sabhlok. “It will help to ease into areas of usage by adopting a tiered approach to expansion and, at each step of the way, review the results against standards, compare notes across the development organization, and course correct where necessary. This can avoid the accumulation of technical debt in the long run.”

“IT leaders should first ensure that additional resourcing and training are in place to both upskill existing teams and hire new talent as required,” said Quillin. “This includes adding observability tools that can identify new and existing forms of architectural and LLM drift. Finally, to always be ready for any new, rapidly emerging future requirements and opportunities, IT leaders should invest in best practices and tools that will proactively keep their app architectures clean and ready for the ‘next big thing.’”

How AI Could Decrease Technical Debt

On the other hand, some believe that if used properly, the contrary is true — that new AI technologies will actually reduce technical debt. “The rise of LLMs, frameworks, and APIs, when applied properly together, will actually reduce technical debt,” said Ashwin Rajeeva, CTO and co-founder of Acceldata. “Code discovery and reuse is more easily accomplished with LLMs.” He points to the ability of this technology to identify duplicative code and consolidate it into a more streamlined interface.

Automated documentation generation is another area where LLMs could help improve code discovery and reduce duplicative efforts. In addition, LLMs could also help reduce the prevalence of code that doesn’t follow company standards, said Rajeeva. “It is much easier for LLMs to understand the company standards by learning from good coding practices and applying it to newly written code.”

Others also see the power of using generative AI to enhance automation across the board. “AI tooling is probably going to help the most in automation, velocity to transform old codebases, and empowering non-coders to accelerate the adoption of new business processes into their workflows,” said Orlandini.

Final Thoughts

AI is a multifaceted subject, and its use could go one of two ways. On one hand, if not used correctly, generative AI could only add to the surmounting technical debt pile. To avoid this outcome, organizations will likely need guardrails on when and how to use LLMs, said Rajeeva. He also advocates “making it a best practice within the corporation with sufficient training for newer developers.”

Then, on the other hand, generative AI could be utilized to respond to technical debt. It has the awesome potential to unlock automation gains and enhance employee satisfaction and retention, especially within older industries that may be behind the technological curve, said Kohli. “With AI and now generative AI, intelligent automation can deliver on all those promises such that it is fast becoming a new pillar of enterprise architecture.”

So, take the new wave of AI with a grain of salt. It will open up immense solutions but will also come with a significant degree of technical debt. Thankfully, there are ways to manage debt caused by AI in a way that can benefit operations and push development objectives forward.

January 25, 2024
Swimm Adds Generative AI Chat Tool for Documentation

Swimm today announced it has added a chat tool that enables developers to use natural language to surface insights into code.

Company CEO Oren Toledano said the /ask Swimm tool makes it simpler to launch queries that enable developers to better understand how code was constructed.

The /ask Swimm tool aggregates documentation along with other related data to surface factors that are not evident in the code itself, such as documentation of business decisions, product design considerations and decisions concerning why specific architectural choices were made. It automatically captures and updates code-related knowledge to provide a continuous feedback loop as code, documentation, files and repositories are created and updated to provide deeper insights into code that goes beyond what might have been actually documented.

The overall goal is to provide a mechanism that enables developers to understand how and why code was constructed within the context of an integrated development environment (IDE), said Toledano.

Swimm has previously made available a platform that uses generative AI platform to create a static analysis of documentation. That capability should make it simpler for organizations to track documentation at a time when generative AI platforms such as ChatGPT are exponentially increasing the amount of code being written. In theory, those platforms should be able to also create documentation for that code, but there will still be a need for tools to track and analyze it.

In the longer term, Swimm plans to continue to extend its usage of generative AI to provide deeper levels of insights across entire applications and software ecosystems to address that challenge, noted Toledano.

In the short term, however, it’s clear that AI is making developers more productive than ever. In fact, the volume of code being generated might soon overwhelm existing DevOps pipelines and workflows. Most DevOps teams will need to revamp those workflows as the pace at which more applications than ever are being developed and deployed faster than ever.

It’s still early days as far as generative AI adoption is concerned, but it’s clear many developers are already using it to write code. How much of that code makes it into a production environment is difficult to determine, but a lot of that code is going to be of varying quality. A general-purpose AI platform such as ChatGPT was trained using code collected from all across the web. The code generated by these platforms is only as good as the examples used to train it, all of which were created by human developers who may have made mistakes, such as including code that has known vulnerabilities.

Regardless of whether code was generated by a machine or a human, the probability that documentation will need to be analyzed as part of a code review process is always high. The challenge and the opportunity is to determine the best way to apply AI to a longstanding challenge that is now being exacerbated by the existence of AI tools that, with each passing day, are only becoming more accessible to developers.

January 17, 2024
No, You Don’t Want to be a Prompt Engineer

Prompt engineering became a thing in 2023 and will stop being a thing in 2024.

We’re already seeing signs of its decline, but let me set the stage for those only vaguely familiar.

A prompt engineer is someone who knows how to get an AI (Normally a chatbot, but honestly, any of the raft of generative AI solutions out there for art, literature, code generation, etc.,) to do what is necessary. While at first, no one was really thinking about this, it quickly became clear that getting a quality result required some massaging in the same sense that getting a quality result from a search engine requires a specific ordering of keywords and limiting of possible results. So, as time went on, those who were better at it started being referred to as ‘prompt engineers.’ I am not certain of the genesis of the phrase, but it honestly doesn’t matter for this discussion. The name became a title, and it got popular; there were some crazy claims about exorbitant salaries for a wordsmith that made it attractive to many.

AI Tools

For some of my work, we use subcontractors to do technical research. Normally, we use technical people and have them turn out results we know will be topically close, even if they didn’t understand the specific technology when they started researching for us. On occasion, we have tried people with less technical backgrounds, and the results are definitively lower quality. With the advent of decent AI tools, we have done some research on using them for this work and have discovered that they allowed less technical people to generate roughly the same quality of results as the more skilled folks.

What this approach doesn’t do is raise either of these researcher types to the level of being able to do something with the results or understand the market well enough to determine results. And we see this across the board. In our development/scripting work, we use generative AI to help, but it doesn’t write finished code complete with the context of our tools/environment. We do that afterward, along with validating the code it does produce. My other business is in publishing and uses a lot of art, so we have contacts with a fair number of artists. Our art director uses generative AI for inspiration but not creation. She asked the other artists we use to follow that example at least until the courts work out IP law about generative AI, and across the board they came to the same conclusion. Some artists are using it for personal/learning purposes, but most AI-generated art is not being generated by professional artists. That has the usual implications for professional artists, but they’re used to being under-appreciated and being underpaid for their work … whether it’s an AI depressing costs versus an amateur doesn’t matter much in context.

Learn it and Use It

Prompt engineering will steadily become less necessary due to simple forces. Just like people learned the ins and outs of search engine prompts, they will learn to do so with AI in their space. As the AI gets more training/specialization, it will get easier to use, and at the same time, it will get more accurate. So, my advice is the same as with all automation tools. Learn it, see how it fits into your organization and use it to your advantage. But it’s not a career; it’s an automation tool. As time goes on, it will be able to do more and more, and maybe one day we’ll be able to say, “Hey Google, write me a CRM that works with our billing system.” But for the foreseeable future, focus on coding and use it to do some of the grunt work.

The only caveat I have also offered before. Go out of your way to hire and train good entry-level people. The weakness of AI doing a lot of the grunt work is that the “learning period” is shorter, so you do need to consciously say, “What is necessary to debug and/or manage our systems?” and make certain your entry-level people get that – because the day-to-day that they have traditionally trained in is far smaller.

And use it to create and maintain even better systems in the new year! I am stoked about the possibilities it opens up, and you should be, too. If I don’t have to write and debug that for loop, just generate, review and test, I’m thrilled. We should all be happy to use that time on more productive things than entry-level code.

January 17, 2024
Best of 2023: Copilots For Everyone: Microsoft Brings Copilots to the Masses

As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.

Microsoft has been doing a lot to extend the coding ‘copilot’ concept into new areas. And at its Build 2023 conference, Microsoft leadership unveiled new capabilities in Azure AI Studio that will empower individual developers to create copilots of their own. This news is exciting, as it will enable engineers to craft copilots that are more knowledgeable about specific domains.

Below, we’ll cover some of the major points from the Microsoft Build keynote from Tuesday, May 23, 2023, and explore what the announcement means for developers. We’ll examine the copilot stack and consider why you might want to build copilots of your own.

What is Copilot?

A copilot is an artificial intelligence tool that assists you with cognitive tasks. To date, the idea of a copilot has been mostly associated with GitHub Copilot, which debuted in late 2021 to bring real-time auto-suggestions right into your code editor.

“GitHub Copilot was the first solution that we built using the new transformational large language models developed by OpenAI, and Copilot provides an AI pair programmer that works with all popular programming languages and dramatically accelerates your productivity,” said Scott Guthrie, executive vice president at Microsoft.

However, Microsoft recently launched Copilot X, powered by GPT-4 models. A newer feature also offers chat functionality with GitHub Copilot Chat to accept prompts in natural language.

But the Copilot craze hasn’t stopped there—Microsoft is actively integrating Copilot into other areas, like Windows and even Microsoft 365. This means end users can write natural language prompts to spin up documents across the Microsoft suite of Word, Teams, PowerPoint and other applications. Microsoft has also built Dynamics 365 Copilot, Power Platform Copilot, Security Copilot, Nuance and Bing. With this momentum, it’s easy to imagine copilots for many other development environments.

Having built out these copilots, Microsoft began to see commonalities between them. This led to the creation of a common framework for copilot construction built on Azure AI. At Build, Microsoft unveiled how developers can use this framework to build out their own copilots.

Building Your Own Copilot

Foundational AI models are powerful, but they can’t do everything. One limitation is that they often lack access to real-time context and private data. One way to get around this is by extending models through plugins with REST API endpoints to grab context for the tasks at hand. With Azure, this could be accomplished by building a ChatGPT plugin inside VS Code and GitHub Codespaces to help connect apps and data to AI. But you can also take this further by creating copilots of your own and even leveraging bespoke LLMs.

Understanding The Azure Copilot Stack

Part of the Azure OpenAI service is the new Azure AI Studio. This service enables developers to combine AI models like ChatGPT and GPT-4 with their own data. This could be used to build copilot experiences that are more intelligent and contextually aware. Users can tap into an open source LLM, Azure OpenAI or bring their own AI model. The next step is creating a “meta-prompt” that provides a role for how the copilot should function.

So, what’s the process like? Well, first, you create a new Azure OpenAI resource. Then you add a data source, which can be as simple as a PDF file or Word document. Next, you can then choose to have the generative AI augment responses or limit itself to the supplied data. Then, the service is exposable through an API to be injected into any application format. The result could be a chat assistant that loops into private data and works well with your own apps and data.

The copilot stack consists of the frontend design, an orchestration layer and a foundational AI model at its core.

Microsoft has also built tools to keep private data safe and encrypted throughout the AI orchestration process. “AI orchestration involves grounding, prompt design and engineering, evaluation and AI safety. These are kind of the core fundamentals for how you create great copilot experiences,” said Guthrie.

You Get a Copilot, and You Get a Copilot and …

The surge of copilots comes at a time when many companies are integrating LLMs and generative AI into their offerings. These powerful AI additions have the ability to accelerate low-code development with advanced auto-complete features. And the Azure OpenAI service looks to be a promising platform to quickly spin up custom copilots.

“AI is going to profoundly change how we work and how every organization operates, and every existing app is going to be reinvented with AI, and we’re going to see new apps be built with AI that weren’t possible before,” said Guthrie.

By constructing their own copilots, organizations will continue to customize the experience with generative AI and give it more context into specific domains to solve unique pain points. And integrating copilots into the realm of end-user interfaces will help open up accessibility to AI in a broader sense.

“Software has eaten the world; now it’s AI’s turn,” said Thomas Dohmke, CEO, GitHub.

January 4, 2024
Digibee Adds AI Tool to Accelerate Migration to Integration Platform

Digibee is leveraging artificial intelligence (AI) to make it simpler to migrate to its integrated platform-as-a-service (iPaaS) environment by converting code into a JavaScript Object Notation (JSON) format.

A translator tool created by Digibee invokes large language models (LLM) to convert that code into a JSON file that can then run on the Digibee Integration Platform.

Initially, that capability is being made available to facilitate migrations from the Mulesoft integration platform made available by Salesforce, with support for other legacy integration platforms planned for 2024.

Digibee CTO Peter Kreslins Junior said this approach reduced the level of time and effort required to switch platforms by using generative AI to map and refactor legacy integration code. That refactoring process doesn’t completely automate the reverse engineering of workflows, but it does substantially reduce the total cost of switching platforms by, for example, automating data migration and simplifying the transfer of configurations, he noted.

Generative AI also eliminates the need to rely on documentation or maps that either may be limited or non-existent, added Kreslins. Instead, all the integration patterns used can now be automatically discovered by an LLM, he added.

IT teams will still need to vet and test the code created by the translator tool, but overall, the speed at which any migration effort is made should be accelerated, thanks to the rise of generative AI, in 2024. Many organizations are locked into various platforms today simply because the effort required to rework the code that has been created to extend them has required either too much effort on their part or the contracting of expensive consultants. A migration project that might have taken a year or more can now be accomplished in a few days, said Kreslins. The overall goal is to reduce the current level of pain IT teams experience when migrating between platforms, he added.

It’s not clear if the rise of generative AI platforms that make converting code into other formats and languages simpler will drive a wave of migrations, but there are many organizations relying on legacy platforms that are not nearly as easy to use as more modern alternatives. As the time and effort required to make those transitions is reduced, more IT teams will be, at the very least, more open to considering their platform options. The providers of those legacy platforms will naturally have to invest in new capabilities much faster to discourage customers from migrating to another platform.

In the meantime, the amount of code that will need to be integrated is about to exponentially increase as developers rely more on LLMs to generate code. The downstream implications of that increased pace of application development for DevOps teams are profound. It’s not clear to what degree those teams can increase the level of scale at which they today manage application deployments.

The one thing that is clear is there is no going back because, so far as application development and deployment are concerned, the proverbial AI genie is out of the bottle.

December 20, 2023
KubeCon: DevSecOps and Generative AI

Speaker 1: This is Techstrong TV.

Alan Shimel: Hi, everyone. Alan Shimel back here at Chicago KubeCon, North America 2023. If you’ve watched our coverage of past KubeCons or RSA or even sometimes on Techstrong TV, you know this fellow right here, he’s my friend, David DeSanto. David, is the… Is it Chief Product Officer?

David DeSanto: Chief Product Officer.

Alan Shimel: Yep, at GitLab. And GitLab always has something doing, especially if it’s the 22nd of the month, right?

David DeSanto: Correct.

Alan Shimel: How many consecutive 22nd of the month releases are we up to now? Do you know?

David DeSanto: Over 150. I know that.

Alan Shimel: Over 150.

David DeSanto: Yeah.

Alan Shimel: But GitLab is also a big community member of CNCF, as well as a bunch of the other Linux Foundation-

David DeSanto: Absolutely.

Alan Shimel: Foundations as well. And David’s always a wealth of information. So David, I hope I didn’t put too much pressure on you, but welcome.

David DeSanto: It’s fine. Thank you for having me.

Alan Shimel: Always a pleasure. So let’s first talk a little bit about the show.

David DeSanto: Yeah, absolutely.

Alan Shimel: GitLab is here. What are you seeing, hearing? What are you thinking about?

David DeSanto: Yeah, so I would say this probably is no surprise to everyone who’s watching, but this year it’s all about security and AI. And sure enough, at GitLab’s booth here, we’re booth M1, it’s also about…

Alan Shimel: Security and AI.

David DeSanto: Security, AI, and visibility.

Alan Shimel: So we’ve been talking a little bit about security and AI too. Surprise. And one of the… We actually, just a quick plug. We actually have a great virtual event coming up in December on AI, and it’s sort of where does AI meet, rubber meet the road. And I want to turn that around to you, David. Look, AI has tremendous potential to help, I think, with security.

David DeSanto: It does.

Alan Shimel: I don’t know how much of it is real today. I think a lot of it will be real, certainly. But what’s real today? No holds barred. What’s your take?

David DeSanto: Yeah, so maybe we’ll start with the AI part of that and then tie it into security. So I always look at new technology through the lens of my parents. They’re both much older than I am. And to them, when I showed them AI, it was magic because to them-

Alan Shimel: It really was.

David DeSanto: How is that happening, right? But then I also had to help ground that in reality for them to say, “Well, this is generative AI. It works by predicting the next word based off the series, right?”

Alan Shimel: Putting one foot in front of the other, right. It’s not sentient.

David DeSanto: When you look at it though, some people have thought it’s sentient.

Alan Shimel: No, your parents, I go by my wife’s sister, she’s 74, and yeah, she thought there was a little person in there, but yeah.

David DeSanto: Yeah, and actually just as a quick note, and the viewers will love this, but I introduced them to it at my sister’s wedding earlier this year. I needed to do a toast. So I had Bard generate the toast for me. And they were like, “How is this happening?” Right?

Alan Shimel: They’re beautiful, I hear you.

David DeSanto: But to kind of ground in reality and you realize how generative AI works, you have to then ask yourself what are the best use cases to apply it to? It’s not going to be able to tell you how many pages there are in a book, how many words are in a book. Those aren’t things that are questions it can build, right? But it can help you with tasks that can sometimes be mundane. It can help you with understanding something better. And that’s how I kind of tie it back to AI. So when GitLab started applying AI to our platform, we started in 2021 with code review because that was a huge pain point helping customers get through code review. The next area we looked at was actually security.

Alan Shimel: Really?

David DeSanto: And the reason why we did that is that GitLab is, in essence was a developer platform when we started, we’re an enterprise DevSecOps platform today.

But it was about helping developers be more effective. And developers don’t start their day off by saying, “You know what? I want to write a zero day vulnerability in the application today. I want to make it super vulnerable.” Right? They want to write really good code. And so we said, “We’ve helped shift security left. How do we help now those developers understand that security they’re seeing?” And so we launched at RSA this year, explained this vulnerability where it actually explains the vulnerability, natural language using AI gives them an example of how that vulnerability is exploitable. And it gives them an example of how to resolve it. And those code snippets it shows are in the language the developer’s writing it. And so when I think about AI and security, I think about it making security truly approachable. And if you can do that, then everyone starts to win.

Alan Shimel: So I would give you guys the Oscar for best use of AI, right? Because I think that’s where it shines. I had a very similar experience with me. So I’m not a coder.

David DeSanto: And I claim to be one, but it’s been a decade plus since I’ve run any code.

Alan Shimel: But you know what? Try putting a little code into your generative AI chatbot and say, “Explain this,” or, “What does this mean?” And it goes through line by line and tells you, “That’s what this is for. This is what it’s calling, this is what it’s doing.” And you can be a complete neophyte around let’s say Python or name your language, wherever you want to do it in. And it really teaches you how to code.

David DeSanto: It does. It does.

Alan Shimel: Line by line, syntax by syntax. So to me, that’s a great use of AI. That’s what we should be using it for. It’s not Google search. A lot of people I know, they think it’s Google. It’s not Google search.

David DeSanto: No.

Alan Shimel: But those kind, and it’s good for writing toasts for a best man or whatever, or an anniversary as someone in my office I know used it and his wife called him on it. But anyway, it’s really great for learning tool. Now, when we talk about AI and security, though, there’s an evil dark side to it.

David DeSanto: There is.

Alan Shimel: Which is how do you secure AI?

David DeSanto: Yeah. And so I love that question. So the first time we spoke, which now is four or five years ago, the question you asked me is, “GitLab and security, explain that.” And we talked about why we’re taking that journey. And today, GitLab is recognized as a security player in the space.

Alan Shimel: Absolutely.

David DeSanto: And it’s one of those things that we are very proud about because we helped define what shift left actually meant. And so if you bring this back to AI, the one thing I always tell people is that, yeah, code generation can be scary. You’re not sure, is it writing good code? Is it writing functional code? But if you’re using GitLab, that code gets committed back into your CICD pipeline and the GitLab security scanners run. And now you know that vulnerability has been resolved and nothing new has been introduced. And so if you pair AI with the right things around it, it can be truly powerful. But you’re right, if you’re not doing that and you’re thinking it is the person living inside the machine, you give a little too much trust to it today. AI is not a hundred percent there yet.

Alan Shimel: Absolutely. Well, so that’s the next segue. Don’t give up any secrets. Let’s not get in trouble. You’re a public company now, but where do you see AI going with GitLab specifically over the next, let’s say between now and RSA in May or something like that?

David DeSanto: Yeah, great question. So fortunately, we’re a very transparent company, so I can talk about roadmap and not get in trouble, but thank you for making sure that I don’t get myself in trouble with legal.

Alan Shimel: Or me.

David DeSanto: So where we see AI going with GitLab is there’s really two paths that we’re exploring. The first is GitLab Duo Chat. We released it back in June, it continues to mature. We’re seeing people gravitate towards wanting to talk to chat and not necessarily click a button in the UI or engage with AI that way. And it makes sense. If you think about it, we’re humans, you and I are talking, we’re having a conversation. That’s how we’d want to engage with it. And that’s why the chatbots are so popular.

And so what we’re looking to do is continue to take what we’ve released. We have 14 AI features today.

Alan Shimel: Really?

David DeSanto: Yeah. Everything from, GitLab is for the entire SDLC. So we have AI assistance in planning, coding, securing, deploying, the whole way across. And so we’re slowly moving those features into chat. An example that explained this vulnerability is a great example. So that’ll be available in chat here shortly. And the reason why we’re doing that is the natural thing is, “Well, help me resolve it. Here’s my code.” And so we want to help AI be not just a pair programmer for the developer, but be that trusted partner that can help write more secure codes. So I would say over the coming several months going into RSA, you’ll see a lot of AI and security and our Duo Chat feature coming together.

Alan Shimel: I love it.

David DeSanto: The other area is our-

Alan Shimel: Wait, hold on one second.

David DeSanto: Oh, sorry. Yeah, go ahead.

Alan Shimel: I want to make sure Duo Chat, not everyone out here is going to know Duo Chat.

David DeSanto: Yeah, so perfect. Yeah. So GitLab Duo is the name of our AI workflows within the GitLab platform.

Alan Shimel: Got it.

David DeSanto: The name is based off the fact that it’s you as a user, plus AI makes the AI dynamic duo.

Alan Shimel: A duo. Okay.

David DeSanto: It also helps that I’m a huge Batman fan and dynamic duo has that in it.

Alan Shimel: All right. I love it. I love it.

David DeSanto: I didn’t tell marketing that though.

Alan Shimel: No, no, no.

David DeSanto: No, no. No. Hopefully they don’t watch this. They’ll now know my secret.

Alan Shimel: What does marketing ever? I’m only kidding. I’m only kidding. I love my marketing people.

David DeSanto: So that’s the name of the suite of those AI workflows. And so there are things like explain this vulnerability. You’re explaining of codes for, we actually haven’t explained this code.

Alan Shimel: I love it.

David DeSanto: We’ll explain the code. We have our code suggestions, which is going to generate either code completion, your cursors in the file, it’s reading what’s before and after the file’s open and recommending code.

Alan Shimel: I love that.

David DeSanto: Or code generation where you can actually do comment decode, that’s actually the most popular for our users. As well as things in helping you do your planning, summarizing conversations, helping with CI configurations and beyond. So that’s what GitLab Duo is, and Duo Chat is part of that. And so that’s the chatbot you can launch within GitLab’s UI.

Alan Shimel: Now, is there a way to kind of implant this into your IDE, or?

David DeSanto: And that’s the last thing I was going to mention that we’re working on. Yeah. So the other thing-

Alan Shimel: You didn’t pay me to say that.

David DeSanto: I did not.

Alan Shimel: Okay.

David DeSanto: I did not.

Alan Shimel: Go ahead.

David DeSanto: It’s almost like you read our GitLab issues and read our roadmap.

Alan Shimel: We may not coordinate our wardrobes. But we do-

David DeSanto: We didn’t. I do wish I would’ve worn the fall shirt today.

Alan Shimel: Next time.

David DeSanto: We will. Next time we’ll do that.

Alan Shimel: All right.

David DeSanto: But anyway, we’re going to be bringing Duo Chat to the IDEs. So today we have it as an experimental feature and we’re trying it at GitLab today to make sure it works. But it’ll be available in all the IDEs that we support today, including the JetBrains Suite, Visual Studio, VS Code, Neovim, and the GitLab IDE.

Alan Shimel: I love it. I love that. Fantastic.

David DeSanto: But by the way, that’ll also then bring those same actions that are in the GitLab UI into chat in your IDE, so you can actually explain the vulnerability, explain the code, all those things directly where you’re working. You don’t have to leave your IDE to get that.

Alan Shimel: While we’re on this subject. A lot of people out here, look, there’s so much, there’s so much controversy around AI. Is it going to kill off humanity? Is it going to take everyone’s jobs away? Is it going… My feeling is, it’s not… Will it do some jobs? Yeah, but for the most part, it’s going to make a lot of people who do these jobs 10x more effective. How do you view it, and how does GitLab view this, in a world where chat and Duo Chat is built in and AI is built into the entire GitLab SDLC pipeline.

David DeSanto: Yeah.

Alan Shimel: Does it really 10x these people?

David DeSanto: So that’s our target. And to give a little context for everyone, GitLab Ultimate, which is our top tier, includes every future GitLab has including AI. We did a survey of our customers. We had an independent company do that. They found that people using GitLab Ultimate get a 7x boost in efficiency for their organization.

Alan Shimel: So you’re over halfway there.

David DeSanto: Yeah. And an ROI within six months of what they paid for it. And so when we asked ourselves, “How does AI help?” We set ourselves a 10x goal. So continuing to accelerate that efficiency. And our customers using GitLab today and GitLab Duo are talking about that. They’re talking about getting through code review faster. They’re talking about resolving vulnerabilities faster. And to the first part of your comment, we don’t see AI as a replacement for your team members. We see it as helping them be more effective, removing some of the mundane tasks that can slow them down and unlocking their true potential.

Imagine a world where your developers don’t have to worry about doing things like creating the template for the code and now they can just go in and write the actual strategic parts. And same thing across the entire software development lifecycle. I personally benefit from Duo almost on a daily basis, in that as the CPO of the company, I get pinged on our GitLab issues or GitLab Epic or whatever, and someone will say, “What are your thoughts on this?” And I’ll look at it and I go, “How long is this?” And I’ll go up and say, “PDF it.” And it’s like it’s a hundred pages, the issue’s been open eight months.

Alan Shimel: Summarize.

David DeSanto: Customers are coming, yeah. So I say-

Alan Shimel: I do the same thing.

David DeSanto: “Summarize the discussion,” right?

Alan Shimel: We’re connected. I do the same thing.

David DeSanto: And so that’s where I see it. That made me more efficient. I now didn’t spend two hours doing that.

Alan Shimel: I’m doing the same exact… As CEO at Techstrong, I’m inundated. Inundated with content. “Should we publish this? Is this the right tone? Here’s the abstracts.” And a lot of times, and I don’t mean to sound callous, but I just don’t have the time to go through it all. But a summary, man, just give it to me in five bullets.

David DeSanto: Right. And by the way, the other part of that that I use is that it could also help generate descriptions and tasks. And so-

Alan Shimel: A bot.

David DeSanto: Right? So I’ll just say, “I read all that. I have an action, remind me to do this. Help me create the task for it,” and so forth. But you are the great example, right? Same thing with the developer, a security person. There are other things that your brainpower should be used for and why spend the time doing those things that-

Alan Shimel: I agree.

David DeSanto: Truly are not moving Techstrong forward or that company’s application for it.

Alan Shimel: We’ll come up with a, we have a theme for given event, a virtual or in-person event. We need the abstract. I can have other people write the abstract, but it doesn’t really capture my vision for it. I find just putting that into bot, into a chat, it nails it. But here’s another thing that I think is, and again, for those of you out there who don’t use generative AI yet, you should. Don’t be afraid to give it a nice long prompt or give it a lot of back. Because the more info you give it, the better it kicks back to you.

David DeSanto: It does. It actually goes off feedback and you can like something or thumbs it down.

Alan Shimel: That was another kind of revelation. And I think it’d probably work the same way with code.

David DeSanto: It actually does. So the backend of GitLab Duo is monitoring for the amount of code you keep. So do you accept the recommendation? Do you not? It also measures retention. So is that code still there after you move on? And same thing with the chat. So I’ve actually had to say to it, and there’s a joke on the internet about you have to be polite back to the chatbot. I actually just recommend it because I’ve seen that documentary Terminator with Skynet. I’m just afraid-

Alan Shimel: One day they might come for you.

David DeSanto: You want to suck up to them now, so when they take over, they remember you.

Alan Shimel: They’ll remember it.

David DeSanto: But no, I’ll actually say back, “That’s interesting, but have you thought about this? And here’s why.” And then I see that it actually learns from it because we constantly fine tune models based off that feedback. Right?

Alan Shimel: I agree.

David DeSanto: So, absolutely. Please be kind to the chatbots. GitLab Duo will always be chat to you.

Alan Shimel: Be kind to your chatbots. And be kind to everyone else, right?

David DeSanto: If it is-

Alan Shimel: Especially in this day with everything in our world going on.

David DeSanto: If it’s anything, you should be polite. There you go.

Alan Shimel: If you can be anything, be polite. Non-AI stuff, what’s your impressions of the show this year?

David DeSanto: I’ve enjoyed it so far. RSA conference used to be my favorite conference to go to. I think KubeCon is now tied for first and might sometimes edge it out. And the main reason why is that here is one of the few times I feel like it reminds me of the Black Hat Conference back in the day.

Alan Shimel: Back in the day.

David DeSanto: You’ve got all the, it is not meant to sound negative, but the doers here, right?

Alan Shimel: Yeah.

David DeSanto: You’re having real conversations with developers who are trying to understand operations better, security professionals who are here trying to make sure they’re supporting developers’, operations. Same thing with operations. So you get this really good mix of people and it just reminds me of almost like a large startup, if that’s a good way to look at it.

Alan Shimel: Yeah, no, there’s an energy that’s palpable, that I think works.

David DeSanto: But I do want to just make sure RSA, I still love you.

Alan Shimel: We’ll be there, May 6th-

David DeSanto: I’ll be there as well.

Alan Shimel: Is our DevSecOps. It’s going to be all about AI in the future of security, our event as well as the larger AI event all week. Or the larger RSA event all week. What about re:Invent? You guys are going to be at re:Invent?

David DeSanto: We will be at re:Invent. I will personally not be there.

Alan Shimel: No?

David DeSanto: No. I actually, I hit the point where this is my last whole week of the year. I’ve been gone three weeks a month for this entire year. I want to stay married.

Alan Shimel: I hear you.

David DeSanto: I want to spend some time with my dogs. And so I’m taking December and most of January off from travel.

Alan Shimel: Good for you.

David DeSanto: Yeah.

Alan Shimel: I’m getting a new dog Friday.

David DeSanto: What kind of dog are you getting?

Alan Shimel: I guess I could share it. So we’ve had bulldogs for about almost 30 years.

David DeSanto: Okay.

Alan Shimel: Our last one passed away in May. She was 14 and a half.

David DeSanto: Actually, I think you and I talked about that.

Alan Shimel: Yeah. So it took a little while for us to get back in the, I need a dog, but I need a dog. So it’s a new bulldog puppy. Her name is Kaia, K-A-I-A, which means sea in Hawaiian. Her father’s name is Ocean.

David DeSanto: Oh, wow.

Alan Shimel: And yeah, I’ll post pictures.

David DeSanto: Okay. I think I’ve shared this to you before, but we have two rescues. They’re both Chihuahua mixes.

Alan Shimel: Oh, yeah. Yeah.

David DeSanto: One’s really dumb and the other one’s really smart, so [inaudible 00:18:47].

Alan Shimel: It’s good mix. Yeah.

David DeSanto: I will tell you, when I get home, they’re so excited to see me and they get really sad when I’m packing, so.

Alan Shimel: There’s nothing like the love of a dog.

David DeSanto: I completely agree.

Alan Shimel: No doubt.

David DeSanto: But about the conference.

Alan Shimel: Yeah, we went off there, but go ahead.

David DeSanto: Completely derailed us.

Alan Shimel: Yes.

David DeSanto: So for GitLab, a couple things. So if people are watching and they’re here, they want to see it. If not, check it out after the conference on our website. We brought our product analytics feature to market. We already have some customers adopting it and we’re demoing it. And I wanted to share it here because KubeCon’s also about visibility and you can’t close the SDLC without user feedback. And so similar to how we have observability, just about the application running in production, the value stream analytics and value stream management. This is now user feedback. So as part of GitLab CICD, we can embed telemetry into the application. So when it’s deployed, it can connect back to GitLab and start giving you data about your monthly active, daily active usage, the return rate of those users.

Alan Shimel: This is available now?

David DeSanto: It is available now.

Alan Shimel: Or you’re just demoing?

David DeSanto: No, it’s available. It is available.

Alan Shimel: What do you call that?

David DeSanto: Well, we love our boring names at GitLab, it’s one of our values. So it’s just called GitLab Analytics.

Alan Shimel: GitLab Analytics. Okay.

David DeSanto: Yeah, GitLab Product Analytics to be specific.

Alan Shimel: All right.

David DeSanto: But yeah, we started using it earlier this year for our handbook and we’re able to learn about how our users use the handbook, our employees and team members. And so I’m excited how make this available because other people want to understand how their apps are being used. Long term, we’ll be able to pull in things like retention rate, user adoption journey metrics. But today it’s just those raw metrics like the active usage, return rate, the graphs of how the usage is going. But we’re very excited to be demoing it here. It’s really cool.

Alan Shimel: Absolutely. Very cool. GitLab user analytics.

David DeSanto: Yep.

Alan Shimel: David, it’s always a pleasure, my friend.

David DeSanto: Love our chats.

Alan Shimel: You know what? I spoke to you. We’re going to do another chat with David before you leave for two months with the dogs.

David DeSanto: Yeah. I will be working, for everyone. I’m not taking time, I’m just staying home. I’ll be-

Alan Shimel: Okay. You’re just not on the road.

David DeSanto: Yeah, I’ll be working from home that entire time.

Alan Shimel: I was going to say that’s a lot of vacation time.

David DeSanto: No, no, no. I did have my first big vacation since the pandemic though.

Alan Shimel: Really?

David DeSanto: Yeah, we went to Bar Harbor for two weeks and hiked Acadia National Park.

Alan Shimel: Oh, that’s beautiful.

David DeSanto: Yeah.

Alan Shimel: And when was this?

David DeSanto: End of August. Last two weeks of August.

Alan Shimel: Oh, so it’s still nice there.

David DeSanto: It was very beautiful, but-

Alan Shimel: Not really peak color season there yet. But beautiful time, summer.

David DeSanto: But we’ll definitely can connect because I’m working. I’m just not on the road.

Alan Shimel: But we’re going to have you on Techstrong TV because you guys have more news.

David DeSanto: We do.

Alan Shimel: That we can’t talk about yet.

David DeSanto: Yeah, that’s a good teaser. Everyone tune in.

Alan Shimel: That was my, that’s what we call it.

David DeSanto: For the big news. It’s the hook.

Alan Shimel: That’s what we call it in the industry. My hook.

David DeSanto: Going back to our marketing friends. That’s the hook.

Alan Shimel: All right. Hey, we better take a break here. We’re getting a little loony. We’ll be back. We’re live in Chicago at KubeCon with our friends at GitLab. We’ll be right back.

December 19, 2023
Sourcegraph Launches Cody Generative AI Tool to Write Code

Sourcegraph today made generally available an open source Cody tool that leverages generative artificial intelligence (AI) tools to write and fix code.

Version 1.0 of Cody invokes StarCoder, an open source large language model (LLM) for creating code made available by Hugging Face and a chat tool that makes use of dense-sparse vector retrieval system for code and documentation to invoke GPT-4 Turbo, Claude 2, GPT-3.5 Turbo, Claude Instant and Mixtral-8x7B LLMs.

Designed to be deployed as an extension of VS Code, Cody is available in a free edition as well as a professional edition, dubbed Cody Pro, that Sourcegraph supports. Cody Pro will be free until February 2024, after which it will cost $9 per month. There is also a version for the JetBrains editor available in beta, along with an experimental Neovim offering.

Cody is also integrated with the code search tools provided by Sourcegraph to provide development teams with additional context that, depending on the use case, delivers a completion acceptance rate of 30% or higher for code.

In addition to enabling developers to write code faster, Coy also simultaneously invokes the graph technology Sourcegraph created to provide additional context to a prompt that makes it simpler to surface relevant code by parsing code and producing a graph that captures the semantic understanding of code. Cody, for example, can search the codebase for files and documents that can be used to suggest a rough plan of attack for implementing a new feature or resolving issues that surfaced in a bug report.

Cody can also create, save and share custom commands to make it simpler to reuse the prompts created to write code.
Future versions of Cody will be more tightly integrated with the Sourcegraph universal code graph to provide additional context. In addition, Cody will soon leverage deeper graph context to make autocomplete suggestions based on symbols defined elsewhere.

Sourcegraph CEO Quinn Slack said the combination of graph technologies and LLMs reduces latency while helping to ensure the code generated by the LLM doesn’t produce hallucinations such as type errors and imaginary function names.

In addition, Sourcegraph is committed to applying AI and graph technologies beyond the codebase itself to provide deeper insights into the environment code will eventually be deployed in, added Slack.

It’s not clear how much developers are relying on generative AI tools such as Cody and GitHub Copilot to write code faster, but it’s clear the pace at which software can be written is accelerating. That increased pace of application development will eventually require DevOps to revisit workflows that were not designed to manage codebases that, in the months ahead, will significantly increase in size.

Each developer will most likely have a personal preference for one generative AI tool versus another, depending on the quality of the code generated. The issue is that no matter how good any of those tools are, however, the code generated will still need to be reviewed by a human before it can be deployed in any production environment.

December 14, 2023
Copado Expands Beta of Generative AI Tools for Salesforce DevOps Platform

Copado today announced it has expanded a beta program through which it provides application development teams building applications for the Salesforce software-as-a-service (SaaS) platform access to generative artificial intelligence (AI) capabilities enabled by ChatGPT.

David Brooks, SVP of Evangelism for Copado, said the company, via the usage of vector databases, has exposed instances of ChatGPT to everything in its knowledge base that has been collected over the last 10 years. That approach enables Copado to extend the large language model (LLM) used to train ChatGPT without exposing any data that might be used to one day train the next version of the generative AI platform.

The goal is to enable DevOps teams to generate code and tests faster while simultaneously reducing the time required to handle support calls, said Brooks.

In addition, Copado is using ChatGPT to enable impact analysis and conflict detection to reduce the level of friction encountered when deploying applications in a production environment. That latter capability is critical because, in many instances today, DevOps teams don’t have a firm understanding of how code needs to be deployed, resulting in unnecessary rollbacks of application deployments, noted Brooks.

Generative AI will also automate the creation of documentation, such as release notes for every application deployed, to reduce the overall level of toil DevOps teams experience, noted Brooks.

Copado also plans to provide governance tools that will enable DevOps teams to ensure the code written by an LLM is not based on a hallucination that results when trying to reconcile conflicting data, added Brooks. There will, of course, always be a need for a human to review code before it is deployed in a production environment, he added.

In the longer term, Copado will make it possible for organizations to invoke multiple LLMs to address various functions as they best see fit, said Brooks. In fact, DevOps teams should expect to soon be employing multiple copilots trained to address specific tasks such as writing code, deploying applications, assessing cybersecurity and addressing compliance mandates, he added.
Ultimately, application development and deployment efforts that today require weeks to complete will soon be accomplished within the same day, said Brooks.

At the same time, organizations will also be able to better manage application environments that today operate at levels of scale few IT teams can manually track. Generative AI capabilities, in effect, will make it simpler for DevOps teams to effectively manage application environments at much higher levels of scale, said Brooks.

Each organization will naturally need to determine to what degree they are prepared to absorb that level of innovation, but it’s clear the pace at which applications are built and deployed is about to exponentially increase. The issue organizations will increasingly encounter is that business processes will not be able to absorb the rate at which software can be built and updated.

It’s not clear just yet to what extent generative AI will transform DevOps, but as advances continue to be made, many of the manual tasks and bottlenecks that today conspire to slow down application development and deployment are about to be eliminated in ways that few would have thought possible a little more than a year ago.

December 13, 2023
Atlassian Delivers on Generative AI Promise for DevOps and ITSM

During a virtual Unleash event, Atlassian today announced its generative artificial intelligence (AI) capabilities are now generally available across its Jira and Confluence suite of tools for managing IT and DevOps workflows.

In addition, Atlassian also committed to adding generative AI capabilities to its Bitbucket continuous integration/continuous deployment (CI/CD) platform to enable DevOps teams to review pull requests and leave comments automatically with suggested changes around syntax and code conventions via a natural language interface. Developers will also be able to autogenerate pull request descriptions from commit messages.

Matt Schvimmer, head of products for the Agile and DevOps division of Atlassian, said the capabilities provided by Atlassian Intelligence will reduce the level of toil and stress DevOps teams experience at a time when the velocity at which applications are being developed and deployed only continues to increase.

Atlassian reported nearly 10% of its more than 265,000 customers have already leveraged Atlassian Intelligence since the launch of the beta program. Organizations, for example, can instantly create user stories within Jira Software tickets, change the tone of a customer response within Jira Service Management, create summaries, launch workflow requests via prompts and generate a starting point for a test plan in Confluence.

Natural language capabilities, currently available for Confluence, will shortly be generally made available for Jira. An ability that promises to help demystify company-specific concepts, jargon or acronyms is also available now in beta, with support for Jira Software and Jira Service Management to shortly follow.

There is little doubt at this juncture that AI will be pervasively applied across both ITSM and DevOps workflows. As those advances are made, it should also become feasible to reduce the level of toil these teams regularly encounter. At the same time, the level of scale at which ITSM and DevOps workflows can be effectively managed should also increase substantially.

Each organization will need to decide how quickly to incorporate AI into those workflows, but hopefully, the level of burnout experienced by IT personnel will be sharply reduced as more tasks are automated. Less clear is the impact AI will have on the size of IT teams required to provide those services, but for the foreseeable future, there will always be a need for some level of human supervision.

In the longer term, the war for IT and DevOps talent that has been waged now for multiple decades should diminish as managing these workflows becomes more accessible, noted Schvimmer. Organizations will not need to rely so much on the expertise of a handful of specialists, he added.

Organizations, however, would be well-advised to make sure AI is embraced from both the bottom up and the top down to ensure optimal adoption, said Schvimmer.

In the meantime, IT teams should take an inventory of the processes that are likely to be automated by AI today with an eye toward restructuring teams as more tasks are automated. Ultimately, the goal should be to let machines handle the tasks they do best to enable the IT staff to provide services that deliver more value to the business.

December 11, 2023