Here is a situation most engineering leaders recognize. You roll out AI coding tools. Features ship faster. Developers are more productive. Then, a few months in, you realize something unexpected: the engineers you most wanted to free up are busier than ever. They are reviewing PRs, firefighting regressions, juggling a dozen half-shipped features and the edge cases that came with each one.
AI made them more efficient. It also made them ten times more busy.
The bottleneck did not disappear. It moved. And it moved to exactly the place most teams are least equipped to handle quickly: verification.
This is the problem that CI/CD, in its current form, is not set up to solve on its own. CI/CD is a delivery mechanism. It runs what you give it. If you give it a pipeline that still depends on humans to write tests, review logic, and triage failures, adding AI on the generation side just means the human verification step gets hit harder. You are filling a faster funnel into the same narrow drain.
The fix is not to slow down the generation side. It is to close the loop on the verification side, and to do it inside the pipeline where it actually belongs.
Testing That Waits Is Testing That Fails
The traditional model treats testing as a phase: you write the feature, you test it, then you ship it. That sequencing made sense when features shipped slowly enough for each step to happen on a human timeline. It does not hold when your coding agent can produce a hundred PRs in the time it used to take a developer to write one.
What you get instead is a backlog. Tests that nobody has time to write. Suites that nobody has time to maintain. A regression that surfaces in production because the team was moving too fast to catch it earlier. Or, on the other side, a deliberate slowdown, where teams cap how fast they ship because quality cannot keep up with output.
Neither outcome is acceptable. And yet most teams have been forced to choose between the two.
The alternative is to make testing continuous rather than sequential: test generation, execution, and repair happening automatically at every stage of the pipeline, triggered by the same commits and PRs that already drive your delivery workflow. Not a separate tool someone opens when they have time. Something that runs whether or not anyone asks it to.
The Gap Inside the Pipeline
There is a deeper reason why testing needs to live inside CI/CD, and it has to do with what software actually is in production.
Code is a small piece of it. The real system is code running against database states, third-party API behavior, configuration values, permission layers, cached data, and the unpredictable patterns of real users doing things the developers did not plan for. That environment is what determines whether software works. And tests that only look at code in isolation, without modeling that environment, will miss exactly the bugs that cost the most.
The CrowdStrike outage in July 2024 is the clearest example of what this looks like at scale. A sensor configuration template and the sensor code were each individually valid. Their interaction, a mismatch in field counts that only existed at runtime, crashed 8.5 million Windows machines. Every individual component passed its tests. The failure lived in the space between them.
That space is what verification needs to reach. It requires testing against a simulation of the real environment, not just a reading of the code. And that simulation has to be continuous, running on every commit, not waiting for a scheduled test run or a manual trigger.
What This Looks Like in Practice
When testing is embedded in CI/CD, every PR gets executed before anyone reviews it. Not code-reviewed, which tells you whether the logic looks right. Executed, which tells you whether it actually works. Targeted tests are generated against exactly the code that changed, infrastructure is set up automatically, and the results come back before the PR reaches a human reviewer. The prompt-test-prompt cycle that eats engineering time either goes away entirely or runs between machines.
Coverage evolves as the product does. The agent maps your application, generates production-ready tests, and updates them when things change. Teams that previously spent weeks building and maintaining test suites get that time back.
API changes, which tend to ripple in unexpected ways, get verified against real endpoint behavior, not just schema validation.
The result is that quality becomes a property of the pipeline rather than a checkpoint at the end of it. By the time a PR reaches production, it has already been tested in conditions close to what it will actually face.
The Real Unlock
AI coding tools deliver their full value when the loop is closed. Writing code faster only compounds if shipping code is also faster, and shipping code faster only compounds if you have confidence in what you are shipping.
Closing that gap does not require changing your stack. It requires adding a layer to the pipeline you already have, one that runs continuously, covers every layer of testing, and makes verification as fast as generation.
That is the pipeline that unlocks what AI coding was actually supposed to deliver.