devopsy.kinsta.cloud

Tag: ChatGPT

Survey Sees Wider Adoption of AI Coding Tools Creating More DevOps Challenges

A global survey of 1,100 finds nearly three-quarters (72%) are now using artificial intelligence (AI) coding tools daily, with on average 42% of the code being created with the help of AI.

Conducted by Sonar, a provider of tools for reviewing code, survey respondents also project that the amount of code being generated using AI coding tools will increase to 65% by 2027.

Nearly all respondents (96%), however, said they don’t fully trust AI-generated code to be functionally correct. The challenge is that nearly half (48%) also admit they don’t always review code generated using AI tools before committing it.

Anirban Chatterjee, senior director of product marketing for Sonar, said the survey results make it clear that, at least in the short term, AI coding tools are creating more challenges for DevOps teams as the volume of code being generated continues to exponentially increase.

The most widely used AI coding tools are GitHub Copilot (75%), ChatGPT (74%), Claude (48%), Gemini/Duet AI (37%), Cursor (31%), Perplexity (21%), OpenAI Codex (21%), JetBrains AI Assistant (17%), Amazon Q Developer (12%) and Windsurf (8%). Just over a third 35% are using personal accounts to access these tools.

Additionally, nearly two-thirds of respondents (64%) have used AI agents, with a quarter (25%) using them regularly. Top use cases for AI agents are creating code documentation (68%), automating test generation (61%) and automating code review (57%).

Overall, the survey finds developers are relying on AI coding tools most often to create prototypes and experiments (88%), followed by internal, non-critical production software (83%), customer-facing applications (73%) and mission-critical services (58%).

Survey respondents also noted they perceive AI coding tools to be most effective in helping them to create documentation (74%), followed by explaining/understanding existing code (66%), generating tests (59%) and assisting in the development of new code (55%).

The top concerns developers have when using AI coding tools are AI code that looks correct but isn’t reliable (61%), exposure of sensitive data (57%) and the introduction of severe security vulnerabilities (44%). A full 88% report encountering at least one issue, such as unreliable (53%) or duplicative code (40%).

Less clear is the degree to which AI coding tools are making application developers more productive. The average amount of time allocated to tedious tasks stands at 10 hours per week, mainly because survey respondents are now spending time reviewing AI-generated code versus writing code themselves, noted Chatterjee.

Among responders with less than 10 years of experience, 40% report productivity gains from AI, but two-thirds of those respondents (66%) also said AI code looks correct but isn’t reliable, with 40% noting that reviewing AI code takes more effort than reviewing the code they wrote themselves.

It’s not likely that developers are going to rely less on AI coding tools because of these issues, but until the quality of the code generated using these tools improves, likely, the number of issues that DevOps teams will need to triage and remediate is only going to sharply increase in the New Year. The challenge now is finding the right set of AI tools to reduce all the time and effort that would otherwise be needed to review what is rapidly becoming an overwhelming amount of code.

January 8, 2026
ChatGPT Developer Mode: Full MCP Access with Serious Responsibilities

OpenAI recently released Developer Mode for ChatGPT, and it’s not for everyone. This beta feature provides full Model Context Protocol (MCP) client support, granting complete access to both read and write operations across your connected tools.

But here’s what you need to understand up front: this is powerful, and it’s dangerous.

What Developer Mode Actually Does

Developer Mode turns ChatGPT into an MCP client. If you’re not familiar with MCP, it’s a protocol that lets AI models interact with external tools and data sources. Think of it as a standardized way for ChatGPT to talk to your other applications.

With Developer Mode enabled, ChatGPT can do more than read information. It can write, update, and delete data in your connected systems. That’s the power. It’s also the risk.

The feature is available to Pro, Plus, Business, Enterprise, and Education accounts on the web. You won’t find this in free accounts, and that’s probably intentional.

Understanding the Risks

OpenAI is direct about this: Developer Mode is intended for developers who understand how to safely configure and test connectors. There are three main concerns you need to watch for.

First, prompt injections. When you’re working with external data sources, malicious content could manipulate ChatGPT’s behavior. If your tool pulls in user-generated content or data from untrusted sources, that content could contain instructions that override your intentions.

Second, model mistakes in write actions. ChatGPT isn’t perfect. It can misunderstand instructions, format data incorrectly, or execute the wrong operation. When you’re dealing with write permissions, a mistake can destroy or corrupt your data.

Third, malicious MCP servers. If you connect to an MCP server you don’t fully trust, it could attempt to steal information from your conversations or abuse the access you’ve granted.

These aren’t hypothetical risks. There are real concerns that developers need to take seriously before enabling this feature.

Mitch Ashley, VP and practice lead, software lifecycle engineering, The Futurum Group shares, “To quote Spider-Man Peter Parker’s uncle Ben, ‘with great power comes great responsibility,’ and that is equally true when using MCP servers. ChatGPT Developer Mode must be treated as a production integration surface, not a convenience feature.”

Mitch continues, “General recommendations when using ChatGPT Developer Mode are to scope permissions tightly, vet every connector, and design for failure with retries, rollbacks, and guardrails. Clear tool metadata and strict trust boundaries reduce misfires. And never rely on perfect user behaviors; build security and verification into the use of tools exposed through MCP.”

Getting Started

Enabling Developer Mode is straightforward. Go to Settings, then Connectors, then Advanced, and toggle on Developer Mode. From there, you can add remote MCP servers in the Connectors tab.

The system supports SSE and streaming HTTP protocols. For authentication, you can use OAuth or run without authentication if your setup allows it.

Once your connectors are added, you can manage individual tools within each connector. Toggle tools on or off as needed, and refresh connectors when you update your MCP server to pull in new tools and descriptions.

Making ChatGPT Use Your Tools

Here’s where things get tricky. ChatGPT won’t automatically know when to use your custom tools versus its built-in capabilities. You need to be explicit in your prompts.

The most reliable approach is to name the specific tool you want. For example: “Use the Acme CRM connector’s update_record tool to change the contact status.” When you’re working with write operations, this specificity isn’t just helpful—it’s essential.

You should also tell ChatGPT what not to use. “Do not use built-in browsing or other tools; only use the Acme CRM connector.” This prevents the model from falling back to its default behaviors when you need it to use your custom integration.

If you have multiple connectors that do similar things, set clear preferences. “Use CompanyDB for authoritative data; use other sources only if CompanyDB returns no results.”

For complex operations, specify the sequence. “First call Repo.read_file with the path. Then call Repo.write_file with the modified content. Do not call other tools.”

Better Tool Descriptions Help

The quality of your tool descriptions matters. When you set up your MCP server, write action-oriented tool names and descriptions. Include guidance about when to use each tool and when not to use it.

Add parameter descriptions and enums. This helps ChatGPT distinguish between similar tools, making it less likely to choose a built-in tool when your custom tool is the right choice.

Think of tool descriptions as documentation for ChatGPT. The clearer you are, the better results you’ll get.

The Safety Net

Developer Mode includes a confirmation system for write actions. By default, any tool that could modify data requires your approval before execution.

You’ll see the full JSON payload for each tool call. Review it carefully. Check that the parameters are correct, and the action matches your intent. Wrong inputs can lead to data loss or unintended changes.

You can choose to remember your approval for a specific tool within a conversation. Use this option only when you trust the underlying application completely. New conversations will require confirmation again.

The system respects the readOnlyHint annotation from MCP tool specifications. Tools without this hint are treated as write actions and require confirmation.

Worth the Risk?

Developer Mode fills a real need for teams building custom integrations with ChatGPT. It gives you programmatic access to your tools without building a separate API layer.

But it requires discipline. Test your connectors thoroughly. Use explicit prompts. Review every write action. And only connect to MCP servers you trust.

If you’re not prepared to handle these responsibilities, stick with the standard ChatGPT experience. Developer Mode is a tool for developers who understand the stakes.

October 15, 2025
Report: ChatGPT-5 Coding Gains Come at a Higher Cost

A report published this week by Sonar finds the GPT-5 platform released by OpenAI has the potential to generate better code but at significantly higher costs.

Based on over 4,400 Java tasks, the report finds that depending on which of the four levels of reasoning capabilities that OpenAI now makes available, the overall quality of the code, especially in terms of the vulnerabilities generated, significantly improves.

However, the overall volume of code being generated per task also substantially increases, which creates additional maintenance challenges for application developers that are not going to be familiar with how code might have been constructed in the first place. Overall, the report finds the minimal edition of GPT-5 produces more than twice the lines of code per tasks assigned than the previous GPT-4o edition of the platform.

There are also plusses and minuses when it comes to security. For example, the report noted that higher reasoning eliminates common, well-understood attacks such as “path-traversal & injection” vulnerabilities. However, these are replaced by subtle, harder-to-detect flaws. The percentage of vulnerabilities related to “inadequate I/O error-handling” increases to 44% in the high reasoning mode versus 30% in the minimum reasoning mode.

A similar trade-off occurs with bugs. As reasoning increases, the rate of fundamental “control-flow mistake” bugs decreases significantly. However, the percentage of advanced “concurrency/threading” bugs increases from 20% in minimal mode to approximately 38% in high mode.

Donald Fischer, a Sonar vice president, said the report makes it clear there is a tradeoff between using, for example, the minimal level reasoning capability priced at $22 per month per developer, and the highest level priced at $189 per developer per month. While there is little doubt that AI coding tools can increase productivity, DevOps teams need to make sure they verify the quality of the output being provided based on multiple dimensions, including the amount of cost being incurred, he added.

The latest ChatGPT-5 analysis builds on a previous Sonar report that assessed the code generated by large language models (LLMs) from OpenAI, Anthropic and Meta, which found each model has its own “coding personality” that affects the quality of the code created. In general, the models possess a strong foundational understanding of common algorithms and data structures and can create viable solutions for well-defined problems.

Additionally, the models are highly effective at translating code concepts and snippets from one programming language to another, which makes them a powerful tool for developers who work with different technology stacks, according to the report. However, critical flaws such as hard-coded credentials and path-traversal injections were common across all models. While the exact prevalence varies between models, all the LLMs previously evaluated produced a high percentage of vulnerabilities with high severity ratings.

It’s not clear to what degree DevOps teams have adopted AI coding tools and just how much they trust the output being generated. There is no doubt that developers will be more productive, especially when building prototypes, but how much of the code makes it into production environments is unknown. The one thing that is clear is that some of the productivity gains being made will eventually come at a cost that many DevOps teams may not realize they are already starting to incur.

September 4, 2025
Coding at the Speed of AI: Innovation, Vulnerability, and the GenAI Paradox
Generative AI (GenAI) is reshaping how software is built. Tools like GitHub, Copilot, ChatGPT and Replit Ghostwriter have rapidly become indispensable in the modern development toolkit, promising increased throughput, reduced toil and faster time-to-market. They suggest code snippets, automate documentation, predict bugs, and even guide architectural decisions. We’re entering an era where developers don’t just write code; they collaborate with machines that do it for them.

But this speed comes at a cost: A rising wave of exploitable vulnerabilities baked into AI-generated code.

A paradox is emerging. The same tools enabling rapid innovation also reintroduce legacy vulnerabilities, spread insecure patterns, and inadvertently create fertile ground for attackers. As development cycles shorten, attackers are moving faster, too, using AI to scan for flaws and weaponize zero-days in record time.

This is not a call to abandon GenAI. Instead, it’s a call to reset expectations, policies and developer training to ensure that GenAI remains a co-pilot, not an autopilot, on the road to secure software.

The Rise of AI-Accelerated Development

GenAI has fundamentally altered the software development lifecycle. From automating repetitive tasks to suggesting code and generating documentation, these solutions can assist developers in their attempts to work faster and focus on high-value problem-solving. Organizations adopting GenAI-powered workflows often see productivity boosts and reduced burnout, particularly for junior developers who benefit from real-time guidance.

GenAI is becoming as essential as version control or CI/CD pipelines for many. But this normalization is where the risk begins to grow.

Blind Spots in the Code

AI-generated code often lacks context, particularly around security. These tools train on massive public datasets that include flawed, outdated, or insecure examples. As a result, GenAI may reproduce known bad practices without warning or the developer noticing.

Some of the most common vulnerabilities now appearing in AI-assisted code include:
- Cross-Site Scripting (XSS)
- Cross-Site Request Forgery (CSRF)
- Insecure deserialization
- Hardcoded credentials
- Open redirects
In some cases, GenAI tools have even reproduced critical, previously patched vulnerabilities. For example, code resembling the famous Log4Shell vulnerability (CVE-2021-44228) has surfaced in AI-generated outputs. While rare or accidental, their reappearance suggests a bigger issue in which GenAI lacks the “judgment” to avoid what it doesn’t understand.

And this is just the beginning. As attackers also leverage GenAI to chain vulnerabilities or create polymorphic malware, the time between a zero-day’s discovery and its exploitation continues to shrink.

The Illusion of Trust

A particularly dangerous assumption among developers is that AI-suggested code is “safe” because it’s syntactically correct or comes from a trusted tool. But code that compiles isn’t necessarily secure.

Because GenAI outputs often appear polished, developers may skip critical steps, code reviews, security audits, or proper documentation checks. This false sense of security is especially risky for junior developers or fast-moving teams under delivery pressure.

Even worse are AI hallucinations, known as syntactically valid outputs but semantically nonsensical. When this happens in a chatbot, it’s amusing. In production software, it’s a backdoor waiting to be exploited.

Best Practices for a GenAI World

Security teams and developer leads should resist the urge to treat GenAI as a plug-and-play solution. Instead, organizations must design policies and workflows that integrate GenAI safely and responsibly.

Recommended practices include:
- Always verify GenAI-generated code with linters, static analyzers, and security scanning tools.
- Cross-reference suggestions with official documentation or vetted code libraries.
- Avoid copy-pasting GenAI output into production environments without manual review.
- Incorporate secure coding training that explicitly addresses GenAI workflows.
- Include GenAI in your DevSecOps pipeline, with checkpoints for security and compliance.
Regulations are Catching Up

Recent policy changes highlight the urgency of responsible AI usage. The EU AI Act and U.S. Executive Order 14110 emphasizes human oversight and risk mitigation in AI-generated outputs, particularly in critical infrastructure and software systems.

These policies make it clear. AI cannot be a black box. Developers and security leaders must be able to explain, audit, and validate what AI creates. The burden is even greater for companies operating globally. Failure to implement oversight may lead to vulnerabilities and regulatory violations.

While AI is powerful, the human developer is still the ultimate arbiter of quality and safety. That means organizations must invest in training that teaches developers how to work with AI, not just beside it.

This includes:
- Understanding how GenAI models are trained—and their limitations.
- Spotting common security issues in AI-suggested code
- Practicing defensive programming and critical evaluation of suggestions
- Embedding security champions within development teams to coach and review GenAI usage.
We’re not trying to slow progress — we’re trying to steer it. Innovation without security is just acceleration toward risk.

Final Thought: GenAI is a Tool, Not a Teammate

The conversation around GenAI and security can’t wait. Ransomware actors and cybercriminals are adding AI to their tactics and arsenals. And many developers trust these tools without questioning their outcomes, unaware they can import the same vulnerabilities attackers are actively scanning.

It’s time to shift our mindset. GenAI isn’t a teammate; it’s a tool. Like any tool, it can be misused, misconfigured, or misunderstood.

Security remains a priority in software development lifecycles. Therefore, integrating oversight, education and security checkpoints into AI-assisted workflows can ensure that GenAI’s promise doesn’t come at the cost of safety.
August 28, 2025
Survey Surfaces Widespread Reliance on Generative AI Among Developers

A survey of 885 developers conducted by Docker, Inc. finds reliance on artificial intelligence (AI) to help build applications is rapidly becoming pervasive.

Nearly two-thirds of respondents (64%) are using AI for coding (33%), documentation (29%), conducting research (28%), writing tests (23%) and troubleshooting/debugging (21%). The most widely used generative AI platforms are ChatGPT (46%), GitHub Copilot (30%), and Google Gemini (19%).

Overall, 65% of the developers surveyed said they view AI positively because it simplifies processes (61%) and enables them to focus more on crucial tasks (55%). Less than a quarter (23%) view AI as a threat to their job.

However, nearly half (45%) said there is too much emphasis on AI. Nevertheless, 40% of respondents identified generative AI as the most important trend in software development, followed closely by AI assistants for software engineering (38%).

Dr. Julia Wilson, user experience researcher for Docker, Inc. noted that more senior and full-stack developers tended to view GenAI as most important, whereas more junior developers with less than five years of experience viewed AI assistants for software engineering as being more important.

It’s still early days when it comes to applying AI to software engineering but it’s clear there are plenty of use cases. Most of that usage currently benefits developers. However, as reasoning engines embedded within large language models (LLMs) become more advanced, it’s now only a matter of time before generative AI platforms can orchestrate a series of tasks that previously would require DevOps teams to write scripts to programmatically perform. Instead, the generative AI platform will create the required scripts required. Those capabilities will serve to eliminate a lot of the toil that ultimately conspires to burn out members of a DevOps team, noted Wilson.

It remains to be seen, however, what impact generative AI will have on how DevOps teams are structured. In theory, an existing DevOps team augmented by AI should be able to build, update and manage a lot more applications. In many cases, the need for entry-level developers might be sharply reduced or, conversely, it may turn out that the junior developers can manage tasks that previously required a senior developer. At the same time, AI may ultimately make DevOps more accessible to a wider range of organizations that previously lacked the resources required to hire and retain software engineers.

The one certain thing is AI is not going back into the proverbial bottle. In fact, the next generation of generative AI platforms will be trained on a narrow set of code that has been vetted for quality and security. That approach should reduce the number of instances where a generative AI platform “hallucinates” in a way that produces code that simply doesn’t work or is rife with known vulnerabilities. General-purpose LLMs such as ChatGPT were trained using code examples pulled from across the Web so the current level of the code being generated is inconsistent.

As a result, one short term concern DevOps teams should have is whether developers become too dependent on generative AI platforms that, by definition, are probabilistic rather than deterministic. In fact, it’s often challenging for a generative AI platform to consistently surface the same code when responding to identical prompts.

As is the case with any major innovation, the challenge is making sure that AI is used both responsibly and safely anytime it is being used to generate code.

Photo credit: Thomas Lefebvre on Unsplash

April 16, 2024
Skillsoft Taps ChatGPT to Improve Developer Training

Skillsoft today revealed it plans to add generative artificial intelligence (AI) capabilities to its Codecademy online training platform to provide developers with hands-on experience using ChatGPT to write code.

Kunal Ahuja, general manager for Codecademy consumer business at Skillsoft, said the goal is to give developers a more curated path toward teaching them how to define prompts that will result in better code being generated. These enhancements will be made available to Codecademy Plus and Pro users as well as enterprise customers in the coming months, he noted.

There’s no doubt that developers are relying more on generative AI to write code, but not all the output generated by platforms that are based on a general-purpose large language model (LLM) is going to be consistent. Codecademy provides a more immersive experience that enables developers to refine their prompt engineering expertise, said Ahuja.

AI-enabled capabilities include code explanations and project hints to improve debugging skills, along with other real-time feedback surfaced via a virtual coding assistant. There are also courses such as “Prompt Engineering for Software Engineers” available.

Finally, Skillsoft is also providing access to a mock interview simulator that developers can access to help determine if they are ready for a job interview.

As it becomes more apparent that prompt engineering is a skill application development teams will need to master to build and deploy software, there’s a clear need to teach nearly everyone in IT how to craft prompts efficiently. Ultimately, the number of prompts used to generate code will have an impact on the total cost of building software.

In the longer term, Skillsoft plans to build its own LLM using data it collects via its platform to provide training using an LLM that has been trained using code that has been curated, said Ahuja. One of the issues with a general-purpose LLM is that it is trained using examples of code pulled from across the web that varies widely in quality. That can result in outputs of code that have known vulnerabilities or that simply run inefficiently. Of course, it may be just as likely that a developer augmented by generative AI is, depending on their level of expertise, actually delivering higher-quality code.

Despite those concerns, however, it’s still important to provide developers with the training they need today to remain competitive, said Ahuja.

It’s not clear how much code these days is being generated by machines versus humans, but as the volume increases, the number and size of the software builds that need to be managed is only going to increase. Hopefully, as much AI will be applied to the DevOps workflows used to manage those software builds as there is being applied to writing code, but in the short term, at least, DevOps teams might soon find themselves overwhelmed.

The one thing that is certain is DevOps teams have a vested interest in making sure the code being written by developers is of the highest quality possible. After all, whenever there is an issue with that code, it usually falls to the DevOps team to fix it.

Image source: https://vecteezy.com/programming-background-with-person-working-with-codes-computer-e1679047506780.jpg

February 20, 2024
Best of 2023: Will ChatGPT Replace Developers?

As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.

AI is buzzing again thanks to the recent release of ChatGPT, a natural language chatbot that people are using to write emails, poems, song lyrics and college essays. Early adopters have even used it to write Python code, as well as to reverse engineer shellcode and rewrite it in C. ChatGPT has sparked hope among people eager for the arrival of practical applications of AI, but it also begs the question of whether it will displace writers and developers in the same way robots and computers have replaced some cashiers, assembly-line workers and, perhaps in the future, taxi drivers.

It’s hard to say how sophisticated the AI text-creation capabilities will be in the future as the technology ingests more and more examples of our online writing. But I see it having very limited capabilities for programming. If anything, it could end up being just another tool in the developer’s kit to handle tasks that don’t take the critical thinking skills software engineers bring to the table.

ChatGPT has impressed a lot of people because it does a good job of simulating human conversation and sounding knowledgeable. Developed by OpenAI, the creator of the popular text-to-image AI engine DALL-E, it is powered by a large language model trained on voluminous amounts of text scraped from the internet, including code repositories. It uses algorithms to analyze the text and humans fine-tune the training of the system to respond to user questions with full sentences that sound like they were written by a human.

But ChatGPT has flaws—and the same limitations that hamper its use for writing content also render it unreliable for creating code. Because it’s based on data, not human intelligence, its sentences can sound coherent but fail to provide critically informed responses. It also repurposes offensive content like hate speech. Answers may sound reasonable but can be highly inaccurate. For example, when asked which of two numbers, 1,000 and 1,062, was larger, ChatGPT will confidently respond with a fully reasoned response that 1,000 is larger.

OpenAI’s website provides an example of using ChatGPT to help debug code. The responses are generated from prior code and lack the capability to replicate human-based QA, which means it can generate code that has errors and bugs. OpenAI acknowledged that ChatGPT “sometimes writes plausible-sounding but incorrect or nonsensical answers.” This is why it should not be used directly in the production of any programs.

The lack of reliability is already creating problems for the developer community. Stack Overflow, a question-and-answer website coders use to write and troubleshoot code, temporarily banned its use, saying there was such a huge volume of responses generated by ChatGPT that it couldn’t keep up with quality control, which is done by humans. “Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking or looking for correct answers.”

Coding errors aside, because ChatGPT—like all machine learning tools—is trained on data that suits its outcome (in this case, a textual nature), it lacks the ability to understand the human context of computing to do programming well. Software engineers need to understand the intended purpose of the software they’re creating and the people who will be using it. Good software can’t be built by cobbling together programs with regurgitated code.

For example, ChatGPT cannot understand the ambiguity in simple requirements. Although it is clear that if one ball just bounces and comes back but another bounces and then bounces again, the second ball has traveled further, ChatGPT struggles with this nuance; that nuance will be needed if these systems are ever to take over from developers.

It also has trouble with basic math, such as when it’s asked to determine which is greater and offered a choice between a negative and positive number. ChatGPT confidently tells us a correct summation of the space, but cannot understand that -5 is less than 4. Imagine your thermostats going haywire because the heating kicks on at 40 degrees Celsius instead of at -5 degrees Celsius because the AI program coded it that way!

Pre-trained AI code generation also raises some legal questions with regard to intellectual property rights; it cannot currently distinguish between code that is licensed in a restrictive or open fashion. This could expose people to licensing compliance risk if the AI borrows a prewritten line of code from a copyrighted repository. The problem has already prompted a class action lawsuit against a different OpenAI-based product called GitHub Copilot.

We need humans to create the software people rely on, but that’s not to say there couldn’t be a place for AI in software development. Just like automation is being used by security operations centers for scanning, monitoring and basic incident response, AI could serve as a programming tool for handling lower-level tasks.

This is already happening, to an extent. GitHub Copilot allows developers to use ChatGPT to improve their code, add tests and find bugs. Amazon offers CodeWhisperer, a machine language-powered tool designed to help increase developer productivity using code recommendations generated by natural language comments and code in the integrated environment. And someone has created a Visual Studio code extension that works with ChatGPT.

And one company is testing AI for developers. DeepMind, which shares a parent company with Google, released its own code generation tool, dubbed AlphaCode, earlier this year. DeepMind published the results from simulated evaluations in competitions on the Codeforces platform in Science magazine earlier this month under the headline Machine Learning systems can program too. Headline grammar aside, AlphaCode achieved an estimated rank within the top 54% of participants by solving problems “that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding.” The abstract for the paper says: “The development of such coding platforms could have a huge impact on programmers’ productivity. It may even change the culture of programming by shifting human work to formulating problems, with machine learning being … responsible for generating and executing codes.”

Machine learning systems are becoming increasingly advanced each day; however, they cannot think like the human brain does. This has been the case for the past 40+ years of study into artificial intelligence. While these systems can recognize patterns and increase productivity for simple tasks, they may not always produce code as well as humans. Before we let computers do code generation en masse, we should probably see systems like AlphaCode rank in the top 75% of participants on a platform like Codeforces, though I fear this may be too much for such a system. In the meantime, machine learning can help with simple programming problems in the future, allowing developers of tomorrow to think of more complex issues.

At this point, ChatGPT won’t be disrupting any field of technology, especially not software engineering. Concern about robots displacing programmers is vastly overstated. There will always be tasks that developers with human cognition can do that machines will never be capable of.

December 27, 2023
Infinite Information

November 6, 2023
4 Things to Consider When Integrating Generative AI

Generative AI has emerged as one of the most influential innovations of the past two decades. ChatGPT gained 100 million users in record time, and new generative AI-powered point solutions emerge every day.

Considering the current state of VC funding, the surge of investment dollars in AI companies is remarkable. Microsoft’s $13 billion investment in OpenAI and AWS’ $4 billion investment in Anthropic (who themselves just raised a $850 million Series C round) demonstrate the tremendous excitement around Generative AI and how it will shape the future of tech.

As newer companies like Adept and Cohere are emerging, existing SaaS companies are integrating generative AI capabilities into their products and services. However, in the race to join the hype, many companies are building first and strategizing later. Before diving headfirst into building a generative AI solution, companies should carefully consider a few crucial factors, including model selection, usage tracking, monetization and customer transparency.

1. AI Model Selection: To Build or to Buy?

Companies must first consider whether to build a generative AI model from scratch, or whether to use a pre-existing model from one of the major AI vendors. This decision depends on the level of customization and control desired, as well as the resources and expertise available.

Building a from-scratch model is likely infeasible due to the inaccessibility of the necessary hardware. The GPUs needed for cutting-edge model training and inference are currently only built by NVIDIA, and there is stiff competition and long lead times for any company looking to acquire the product. What’s more, large cloud and AI companies like OpenAI, Google, Meta and Tesla already have built systems with up to 100,000 of these chips. New companies looking to enter the scene would face a steep climb to assemble the infrastructure needed to even begin to compete with the current classes of models.

Small or new companies do not have the funding or time to invest in building their own infrastructure or generative AI models, so a key business decision is to select the best vendor and model for the use case. While ChatGPT is the most well-known, there are other specialized models that can be better suited for certain use cases. For example, OpenAI’s DALL-E is built for image generation, while Anthropic’s Claude 2 is built to perform better with specialized tasks such as legal and math and to handle long-form inputs.

Regardless of the AI vendor you choose, you will be charged based on your usage with them, which, by extension, is your customers’ usage. Tracking this usage is critical to your generative AI solution becoming profitable.

2. Metering AI Service Usage: The Key to Success

Implementing usage metering for generative AI services is crucial at both the individual user level and at the global or system-wide level. This is because usage tracking is valuable to all business functions and is a critical element of any modern PLG strategy.

For example, customer-facing teams can leverage usage data to proactively track account health and to contextually recommend new solutions and approaches. Usage history provides valuable intelligence to account teams for up-selling, cross-selling, and contract renewal. Product teams can identify usage patterns and customer segments, as well as areas of friction or improvement for new feature development.

Usage data also plays a major role in developing pricing plans that align with customer expectations and overall market trends. It can be used to inform pricing plan development and optimization to ensure the rates are sensible for the level of consumption and value being realized. Without tracking usage, this becomes a guessing game or a futile exercise in “gut feel.”

Usage data is imperative to calculating your internal costs. Vendors commonly charge according to a usage-based model for these generative services–per word, per token or per generation are some common models. In any case, having accurate, granular usage data is a must-have for calculating internal costs and, in turn, customer-facing pricing.

3. Monetizing Generative AI Services: Usage-Based Pricing Strategy

In the rush to beat competitors to market and offer generative AI services to customers, do not neglect to lay the groundwork for monetization from day zero. It is much easier and more efficient to build with monetization in mind from the start than it is to backtrack and try to monetize a service once it has been offered, particularly when usage-based pricing is involved.

Usage-based pricing has emerged as the optimal pricing strategy in the AI domain. This is because service usage varies over time for each customer. These services are priced by the vendors according to a usage-based pricing model. Since the backend costs of the service are elastic, the frontend, customer-facing pricing should also be elastic.

As a roadmap to monetization, follow these steps:

-Meter service usage to understand segments and patterns

-Calculate internal costs of usage using vendor pricing

-Using the information from steps 1 and 2, build customer-facing pricing

-Track revenue and margins and continue to iterate with pricing as needed

4. Customer Transparency: On-Demand Access to Usage and Billing Data

Prioritizing transparency and clear communication with customers is non-negotiable. Ensuring that usage and billing data can be seen at any point in the billing cycle helps foster trust between the company and its users and eliminates billing surprises when the invoice is delivered.

Adopting Transparent Communication Builds Trust With Customers

Invest in creating a customer-facing application, portal or experience that allows for on-demand access to usage and billing data. The beauty of combining this level of visibility with usage-based pricing is that customers are empowered to pay for only what they use while viewing usage statistics on demand and modulating usage as needed in real-time to keep the spend within an acceptable range.

In practice, usage data is available on a per-query level from the vendor’s APIs. For example, for any generation action, OpenAI will show the precise number of tokens consumed for both the prompt and response, as well as all details around the model type and context needed to identify the per-token rate from the vendor pricing. It is important to collect this data and build a user-friendly way to display it back to customers in an app, portal, or website that allows for on-demand access and browsing.

Conclusion

While planning to build a product with generative AI, take all of the above factors into account. Do not make the mistake of racing to deliver a technology solution without also considering the business implications and monetization strategy. With all the hype, excitement and investment around generative AI in 2023, it is akin to a modern-day gold rush. Don’t show up to the gold rush without a shovel.

October 26, 2023
Will ChatGPT Replace Human Software Developers? Probably Not

Since the release of ChatGPT, there has been a great deal of hype around generative AI and how companies can leverage it to cut costs and democratize software and application development. Naturally, with discussions of cost-cutting and democratization come whispers about what will happen to software developers.

This is a real and valid concern, but software developers’ skills, expertise and creativity are still very much needed.

While generative AI and AI code generation tools like ChatGPT have shown some promise and potential benefits, they are still in their infancy—like many other innovative technological advancements. We also don’t know what scenarios they may present down the road or their true abilities when the technology matures. For instance, how will it integrate with other technologies? We don’t know what will happen when a ChatGPT-generated line of code breaks or needs to be changed. We don’t know if it can provide a novel solution to a unique problem or what security threats it will present.

Given these unknowns, technology executives should think twice about replacing experienced and creative technology talent, such as software developers, with AI code generators.

Will ChatGPT create novel code to solve a unique problem never encountered before? Probably not.

A Tale as Old as Time (Or at Least a Decade)

The technology industry has been searching for and developing new ways to make certain software development tasks much easier and more streamlined for years. One example of this is low-code/no-code.

The notion of simplifying application development and replacing software developers with laypeople (citizen developers) has been around for more than a decade now, as low-code and no-code solutions have grown more popular. These solutions have promised that companies don’t need technical talent to get their software and app projects off the ground.

However, if you look at the impact of these solutions today, their use can result in large amounts of technical debt and almost always require the skill of experienced software developers.

The reason? Building complex software and applications is extremely difficult; it’s an art.

Low-code and no-code solutions have their rightful place and can make things easier if a company is looking to launch a simple app or static web page. These solutions can increase the pace of development and time-to-market and enable everyday people without any development skills to facilitate them. However, they are not actually a complete solution and often overlook aspects of development that a human software developer would typically address. Without a skilled expert involved, low-code/no-code platforms often can’t solve a unique problem a company has.

So, how does this relate to AI code generators like ChatGPT? Here’s how.

A Similar Situation—With One Key Difference

When thinking about their place in the development process, AI code generators are not that different from low-code or no-code solutions. The thinking is that they will also enable non-technical individuals to create software and applications with ease.

Yet, there is one key difference—they promise expertise, too. But is the expertise coming from the AI code generator or the person piloting it? The answer is simple; it is not from the code generator.

There have been examples of companies and individuals that have tried using ChatGPT to build code, and they have appeared to be successful. However, without the input of the individuals using it, it never would have been able to create the code on its own unprompted.

This means that the expertise of skilled, human software developers is still very much needed, especially when building intricate software and apps that require complex functionality such as optical character recognition (OCR), internet-of-things (IoT) connectivity and more.

A Future With Software Developers

Because of the expertise still required to create elaborate and innovative software and applications, AI code generators will not replace software developers anytime soon, if ever.

The technology used in these tools is still in its infancy, presenting many lingering questions and potential challenges. We will see this confirmed by the trials and tribulations experienced by the few companies and executives attempting to replace software development teams with tools such as ChatGPT.

Knowing AI code generators follow a similar path as low-code and now-code solutions, companies and executives should use them as developer enablement tools. These tools should complement the work of software developers by empowering them to focus on the complex tasks software and apps today require.

September 14, 2023