Harness today launched a code repository service that is specifically designed for DevOps teams that are relying on artificial intelligence (AI) agents to generate code.
Martin Reynolds, Field CTO for Harness, said the Agent-Ready Harness Code Repository and AI Code Review service provides an alternative to existing GitHub source code repositories that were not designed to handle the volume of pull requests and updates that are made by AI agents operating at machine speed.
As a result, there are not only more outages but search and file history get slower as indexing falls behind, pull requests pile up faster than anyone can read them, and a permission system designed around a list of developers lacks the ability to identify and track an AI agent that might merge code on its own. The assumption legacy source code management tools make is that a human wrote code and opened a pull request that will be reviewed sometime later.
In contrast, the Harness Code Repository has been tested to handle thousands of simultaneous pull requests and commits from both AI agents and human application developers, said Reynolds. Search, history, and diffs keep working across repositories of any size that can have tens of thousands of branches.
Harness Code Repository and AI Code Review are both designed to be reachable via the Harness Model Context Protocol (MCP) server or command line interface (CLI). DevOps teams can review pull requests by the author’s email instead of an internal ID, see every open pull request across every repository and create, reply to, or resolve comment threads without opening a browser.
AI coding agents can either be assigned specific permissions or they can inherit them from application development teams that have deployed them. DevOps teams can further define what an AI agent can access, merge, or deploy, much like they do for any human software engineer using role-based access controls (RBACs) and policies based on the Open Policy Agent (OPA) framework provided via the Cloud Native Computing Foundation (CNCF).
The AI Code Review then reads a pull request, checks to see what gates are mandatory and prevents any request that fails from being merged. Additionally, differences are grouped by risk rather than by file, so high-risk changes that alter how the software behaves are not lost under a mass of renamed repositories and dependencies. A feedback capability also describes what a change puts at stake rather than noting that a line moved. If the feedback is valid, changes can be merged with a single click.
Any team can start using Harness Code Repository for free. Migrations from another repository, including GitHub, GitLab, Bitbucket and Azure DevOps, only require a few clicks, noted Reynolds.
In general, DevOps teams are starting to realize that a piecemeal approach to agentic AI engineering isn’t going to be sufficient, said Reynolds. Instead, the entire software development lifecycle (SDLC) will need to be re-engineered to manage what is becoming massive amounts of code that is overwhelming existing DevOps workflows, he added.
In the case of Harness, that means commits, review, builds, tests, security and deployments all stem from a single sequence within the platform. The Harness Software Delivery Agent then runs those workflows from commit to production using policies created via a single common engine. All workflows are then mapped in the Harness SDLC Knowledge Graph to provide the context needed for every review. Based on early testing, Harness claims DevOps teams have seen savings of 10,000 hours over the last month alone.
Mitch Ashley, vice president and practice lead for the Futurum Group, said the bottleneck in AI-assisted development has moved to review and approval. Agents produce code faster than teams can verify it, and verification debt builds faster than anyone can hire reviewers, he added.
The key here is still accountability, noted Ashley.
Regardless of the approach to software engineering in the age of AI, it’s now more a question of when rather than if DevOps workflows will be re-engineered. The only thing that remains to be seen now is how long that might actually require.
An AI agent that can call tools is not automatically an unsafe system. The dangerous move is smaller: Letting a workflow turn a recommendation into an external action without a durable decision record.
That distinction matters in DevOps. An agent can sensibly summarize a failed deployment, draft a rollback plan or classify a dependency alert. The moment it opens a change request, modifies a feature flag, creates an incident, messages a customer or publishes a configuration, the system needs more than a high-confidence score.
It needs a change-control gate.
The gate below is a practical pattern for a tool-using agent. It verifies whether the policy is still current, pauses for the required human decision, makes one idempotent outbound request, then confirms the receipt. It is deliberately boring. That is the point.
Figure 1: Illustration of a Workflow Moving Through Policy, Human Approval and Idempotency Gates While a Risky Duplicate Path is Stopped
Why a Confidence Threshold Isn’t Enough
Confidence says something about a model’s answer. It does not say whether a particular action is reversible, who owns its consequences, whether the policy changed while the agent was working or whether the same action has already happened.
For a production workflow, treat those as separate decisions.
Question
What the Gate Should Decide
Is this action permitted now?
Revalidate the current policy and declarations.
Does this action need a person?
Route it to an explicit approver with the exact action payload.
Has this action already been sent?
Use an idempotency key stored before or with the transmission.
Did the destination accept it?
Verify a receipt, job ID or visible confirmation without resending.
This maps cleanly to the NIST AI RMF’s idea that risk management should cover governance, measurement and management throughout the system life cycle — not merely model evaluation. It also fits a practical fact of durable agent runtimes: A paused workflow can restart a node when it resumes, so outbound side effects must be idempotent. NIST AI RMF and LangGraph’s interrupt guidance both make the broader control problem explicit.
The Smallest Useful State Machine
Do not begin with a large autonomous platform. Begin with explicit states that an operator can inspect.
The useful property is not the spelling of the states; it is that ‘approval_required’ and ‘transmitting’ are distinct. A system should be able to show an approver exactly what will happen before it happens, then preserve a record of the result.
Revalidate Policy Immediately Before the Action
The policy check belongs close to the outbound call. A week-old permissions decision is not a preflight check.
This is where an organization can apply its real rules: Production versus staging, maintenance windows, owner groups, regulated data or a mandatory incident commander for customer-impacting work. Keep the policy data outside the prompt. A prompt may explain a rule; it should not become the only enforcement point.
Ask for Approval Using the Actual Outbound Payload
An approval that says, “approve the agent’s plan?” is too vague. Show the action type, destination, change summary and required declarations.
If the user rejects the request, end the workflow. Do not reinterpret a rejection as a request to regenerate a more persuasive plan.
Make the Transmission Idempotent
The awkward failure is an ambiguous one. The receiving system may complete the change while the client times out before it receives a response. If the agent retries blindly, it can create a duplicate ticket, a second deployment or a second message.
Use a stable key derived from the action’s business identity — not a fresh UUID for each retry.
The destination must honor the key, or your own durable store must do so before the outbound request. Either way, do not put an automatically retried network call before a pause node unless that side effect is safe to repeat.
Verify Instead of Resending After Ambiguity
Treat a timeout as an observation problem first. Query the destination with the key, look for the created job or ticket and write the result to the workflow state. Only retry if the system can prove that no action exists.
async function verifyReceipt(key: string) { const result = await findActionByIdempotencyKey(key); if (result) return { accepted: true, receipt: result }; return { accepted: false, issue: “No receipt found; needs operator review” };}
That final status should be ‘submission_unverified’, not ‘failed’. The name forces the right next action: Investigate before sending another request.
A Controlled Simulation: Where Duplicates Appear
To make the trade-off concrete, I ran a deterministic simulation of 10,000 outbound agent actions. Each simulated action had a 9% policy-change state, a 21% non-approval state and 1–3 resume events. This is a controlled illustration of workflow logic, not production telemetry, a reliability claim or a measured industry failure rate.
Control Design
Sends
Duplicate Sends
Policy-Bypassing Sends
Unapproved Sends
Ungated Send
12,114
2,114
1,106
2,544
Approval Only
9,570
1,673
879
0
Preflight Gate
8,691
1,514
0
0
Durable Gate
7,177
0
0
0
Figure 2: Bar Chart Comparing Outbound Sends and Duplicate, Policy-Bypassing and Unapproved Actions Across Four Simulated Control Designs
The notable row is the preflight gate. It eliminated actions that bypassed policy or approval, but it still created 1,514 duplicate sends because resumed workflow events could enter the same action more than once. The durable gate reduced that to zero in this simulation by using a stable idempotency record.
The simulation code and raw summary are included with this submission so that editors or readers can reproduce the numbers. Modify the assumptions before applying the result to your own environment.
Keep the Approval Boundary Narrow
This pattern is not an argument for asking a human to approve every tool call. That usually leads to notification fatigue and workarounds.
Instead, classify actions by consequence:
Action Class
Typical Examples
Default Control
Read-Only
Fetch a deployment record, inspect a log
Allow with audit trail
Reversible Internal
Create a draft change ticket, add a label
Allow or sample review
Material but Reversible
Change a feature flag, queue a rollback
Named approval plus idempotency
External or Hard to Reverse
Deploy to production, message a customer, delete data
Named approval, policy recheck, idempotency, receipt verification
The fastest agent systems make safe paths easy and consequential paths explicit. They do not pretend all operations have the same risk.
What to Test Before Enabling an Agent Action
Run these cases in a non-production environment before connecting a new action to an agent:
A policy changes after the agent produces its recommendation but before the approval click.
An approver rejects the action, then the workflow resumes.
The destination completes the change, but the client loses the response.
A queue redelivers the same event.
An operator tries the same action manually while the agent is waiting.
Receipt verification is temporarily unavailable.
For each test, the expected outcome should be visible in state: Blocked, rejected, submitted, verified or unverified. “The agent probably handled it” is not an operational status.
The Operational Rule
AI agents can move quickly without bypassing change control. The reliable pattern is straightforward: Validate the rules immediately before the action, show the real payload to the right approver, record one idempotent transmission and verify the result before retrying.
That sequence gives teams something better than a promise of autonomy. It gives them a system they can inspect when the answer is ambiguous and the consequences are real.
Raw Results: `research/contribution-gate-simulation-2026-08-21/summary.json`
The simulation uses a fixed seed and synthetic policy, approval and resume-event rates. It does not measure any customer system.
The article’s workflow pattern was implemented and tested in TypeScript as part of an internal contribution-control graph. The sample code is simplified for teaching.
In an enterprise environment, production failures are more often defined as testing failures, but they include other high-impact aspects, such as release readiness, data quality, exception handling and operational approval. However, the missing layer is not just about conducting more testing. It should include a proper, disciplined review process for the validation of business-critical data and release activity.
The problem is not just about the effect of production on payroll, benefits, reporting, reconciliation or other sensitive operational functions. Production issues can also lead to payment problems, downstream reporting errors, audit concerns and operational disruption. Production readiness should be treated as a business-control question, not just as a software-quality question.
What Production Validation Means
Figure 1: Enterprise Release Workflow Incorporating Production Validation
As illustrated in Figure 1, production validation is the operational discipline that sits between ordinary release preparation and moving changes into the live environment․ It exists to ensure that key records‚ release outputs and downstream expectations have been appropriately reviewed before changes are moved into the live environment․
In contrast to pass/fail testing validation‚ production validation includes checks to determine whether the data is complete‚ exceptions are handled‚ differences are worked out and the release criteria are acceptable for production․ Production validation usually requires multiple views of validation results‚ reconciliation sign-offs between files‚ issue tracking results‚ and proof of approval, testing and readiness for operations․
Why Testing Alone Isn’t Enough
Testing is important‚ but it does not answer every question about whether a given code can be released to production․ The code may work in a test environment and still have problems in production․ This can be due to timing dependencies‚ data-state differences‚ downstream dependencies and business controls in the production environment‚ which are not necessarily covered in standard testing cycles․
Therefore‚ with a test-only approach, bugs can be discovered too late‚ making them difficult to fix once they have reached production in a running application․ Production validation addresses this concern by treating release readiness as a formal gate rather than merely an assumption, as summarized in Table 1․
Table 1: Comparison Between Traditional Testing and Production Validation
Aspect
Traditional Testing
Production Validation
Primary Objective
Verify software functionality
Verify operational and business readiness
Environment
Test/UAT environment
Production or production-like environment
Focus
Code correctness and defect detection
Data integrity, release readiness and operational governance
Validation Scope
Functional and non-functional requirements
Business rules, configuration, approvals, reconciliation and exception handling
Outcome
Software passes testing
Release is approved for safe deployment
A Practical Model for Production Validation
A useful production-validation model describes a set of interrelated control practices that build confidence in an upcoming release before rollout (Figure 2)․
Figure 2: Core Components of the Production Validation Process
1. Structured Pre-Production Review
Some or all records and expected outputs may be checked against business rules and operational expectations to determine whether key data changes are complete‚ expected outputs are present and the release appears safe from a business-risk standpoint before the release is approved to proceed to the next stage․
The purpose of this is not only to check technical correctness‚ but also to assess whether the release is sufficiently trusted for use in production․
2. Exception Review and Correction
A good validation solution should provide the ability to generate reports of discrepancies‚ review affected records and ensure that issues are addressed prior to moving the release to production․ For many organizations‚ this is done through exception summaries‚ discrepancy reports‚ reconciliation reports or other review-oriented artifacts․
This is worthwhile because it helps prevent teams from treating exceptions as background noise and reinforces that exception handling is part of the release governance process․
3. Workflow-Based Release Discipline
When production validation is included in the workflow‚ it helps ensure that releases are reliable because the status of issues‚ validations‚ dependencies and approvals is explicit․ This means that a release is ready to go‚ not just because the team says it is ready‚ but because it has reached a valid state․
Incorporating workflow-based governance allows for better traceability of validation activities and reduces the likelihood of skipping important validation activities to meet deadlines․
4. Documentation and Repeatability
Standardized procedures are used to verify releases rather than relying on memory․ Validation checklists and runbooks provide a disciplined approach to repeatable execution of software releases․
Aside from promoting consistency‚ this documentation also provides defensibility in high-risk environments that require teams to justify why they believe a release is safe for delivery beyond the review performed․
5. Manual Business-Level Validation
In critical scenarios‚ manual review still matters․ Some production decisions are subjective‚ requiring judgment above and beyond post-release checks․ Conducting review activities before and after a release‚ comparing expected and actual results and analyzing differences at the business level can help identify problems that a technical verification process might overlook․
Manual validation should not be seen as a weakness because it is often the final check to determine whether moving a release into production is truly safe․
What This Changes Operationally
The value of production validation doesn’t come from adding a single tool; rather, it adds review‚ correction‚ documentation‚ workflow discipline and operational evidence to the release-readiness process for the production system․
Organizations can benefit from this strategy in several ways:
Defective records are less likely to be produced․
Exceptions are identified earlier․
Release approvals become less difficult․
Teams have better traceability into what was reviewed and resolved․
Operational reliability improves for business-critical systems․
This transition is more than a semantic one․ Over time‚ organizations can mature from passive release confidence to production assurance․
Why This Matters in Regulated Environments
For regulated or high-accountability environments‚ a faster release is not enough; demonstrating control is also necessary․ When sensitive records‚ reporting requirements or downstream financial results are impacted by a production change, teams must be able to describe how the change was reviewed‚ what risks were identified and why it was approved․
That is why production validation should be considered a separate operational discipline and not just an extension of testing․ It is not just a question of database controls․ The important question is whether an artifact locked for production has been sufficiently reviewed‚ corrected‚ documented and controlled before being used in production․
Conclusion
Enterprise release failures do not necessarily mean that nothing was tested‚ but rather that the definition of ‘production-ready’ was too narrow․ In data-heavy‚ high-risk systems‚ safe release management requires a wider operating model that includes review processes‚ exception handling‚ workflow governance‚ documentation and business validation of processes and workflows․
That is the missing layer in many release programs․ With a disciplined production-validation approach‚ organizations can reduce the risk of losing valuable operational data before defects lead to financial‚ reporting or regulatory issues․
Tricentis is providing early access to multiple artificial intelligence (AI) capabilities that it is gearing up to roll out later this year via a Tricentis Transform initiative, including an autonomous AI agent, dubbed Aida, that explores web and Windows desktop applications to surface defects, weaknesses and other potential gaps without requiring DevSecOps teams to create scripts.
Additionally, Tricentis is developing a Release Risk Intelligence capability that surfaces release-scoped coverage gaps, prioritizes risks by severity, and launches AI tasks to resolve issues that are informed by context provided by the automated Tricentis testing platform.
Finally, Tricentis is also readying AgentScore, a tool that enables organizations to evaluate AI agents by observing how agents behave in real-world workflows, recommending what should be measured, and generating composite quality scores that include review, block, or ship recommendations.
David Colwell, vice president of AI and machine learning for Tricentis, said the overall goal of the Tricentis Transform initiative is to ultimately make AI application testing a more integrated element of building and deploying any application rather than being an entirely separate function. The end result should be higher quality applications that are inherently more secure, he added.
The launch of the Tricentis Transform initiative comes on the heels of the company’s acquisition of Tabnine, which developed a knowledge graph that AI agents can invoke to discover the relationships between various application components. Armed with insights that are uniquely provided by a testing platform, AI agents are provided with the context needed to automate tasks much more reliably, noted Colwell.
It’s still early days so far as embedding AI agents into DevOps workflows is concerned, but as more code is generated using AI tools, there is a corresponding increase in the amount of pressure on existing pipelines. The first place that pressure is manifesting itself is in the need to review and test code at machine speed. In the absence of that capability, more AI-generated code is being deployed in production environments with predictable results. The number of incidents in production environments that have been traced back to AI code has already begun to exponentially increase, noted Colwell.
Exactly how long it might take to re-engineer DevOps workflows in the age of AI remains to be seen, but it’s apparent that testing will become more integral. The difference is that much of that testing is likely to be conducted by AI agents that are reviewing and analyzing code created by other AI agents. The challenge, in the meantime, is a coverage gap that is widening as the volume of code being generated overwhelms the ability of DevOps teams to test it.
Hopefully, there will come a day when the distinction between creating and testing code becomes so blurry as to be all but meaningless. That doesn’t mean the need for specialists to test applications will completely disappear, but it does mean that any issues that do require their expertise will be worth the time and effort for them, rather than an AI agent, to personally resolve.
Enterprise AI agents need more than stronger models. They need durable execution environments that can coordinate multi-step workflows, survive failures, pause for human review and resume reliably after disconnects or delays.
AI Agents Have Moved Beyond Chat Demos
AI agents work beautifully in demos. A user asks a question, a large language model generates a response and everyone sees possibility. Production systems are different.
Consider what these systems are now asked to do. An agent assessing a change request may pull deployment history, evaluate blast radius, check freeze-window policy, wait for a release manager’s sign-off and then schedule the rollout. An agent adjudicating an insurance claim may extract fields from submitted documents, cross-check them against prior claims, apply underwriting rules and route anything unusual to a human adjuster. An agent triaging a security alert may enrich indicators, correlate against past incidents, assess asset criticality and hold containment until an analyst approves it.
Those three workflows have almost nothing in common at the domain level. They share a shape: Multiple steps, several systems, independent failure modes and at least one point where a person decides. They take minutes or hours, not milliseconds.
The challenge is no longer only intelligence; it is execution. Many enterprise AI conversations focus on model selection, prompt engineering, retrieval-augmented generation and tool calling. Those are important, but they do not answer what happens after the agent begins executing long-running work across multiple systems.
The Problem: Most Agents Are Still Designed Like Request-Response Applications
Traditional AI applications assume a simple request-response model: User input goes in; a model response comes out. That model works well for Q&A, summarization, content generation and search-like experiences.
Enterprise workflows are different. A procurement review agent might parse a contract, compare its terms against a standard template, check vendor risk ratings, invoke a sanctions screening service and produce a recommendation. These steps may execute independently, fail independently and complete at different times.
Once an agent crosses the boundary from answering a question to executing a process, it starts to look less like a chatbot and more like a distributed system. It needs state, retries, coordination, observability and recovery.
Why Stateless APIs Break Down
Teams often start by implementing an agent as a synchronous API. That approach works until the first real workflow appears. What happens if the HTTP request times out? What if the user closes the browser? What if an external system is unavailable? What if a human approval arrives six hours later?
A stateless API can initiate work, but it is a poor place to manage long-running state. Developers quickly end up building custom queues, status tables, retry logic, compensation logic and ad hoc workflow tracking. The hidden cost is not the first prototype; it is operating the system after it begins handling real workloads.
The missing layer is a durable runtime: A workflow engine that can preserve progress, coordinate parallel work, wait for external events and resume execution reliably.
Figure 1: The Same Multi-Step Workflow Under Two Runtimes
A stateless service holds progress in process memory, so a host failure discards completed work and the retry begins again from the first step. A durable runtime checkpoints each completed activity, so a replacement worker replays the history and continues from where execution stopped.
The Key Design Shift: the Workflow is the Application
The most important design shift is this: The model is not the application. The workflow is the application. The model is one activity inside a larger execution graph.
In production-grade agentic systems, the orchestrator coordinates work while specialized agents perform specific tasks. The specialization is domain-specific, but the division of labor recurs: One agent gathers and enriches raw signal, another retrieves reference material, another looks for precedent among prior cases and another evaluates policy or risk conditions.
This design makes the system easier to reason about. Each agent has a focused responsibility, and the orchestration layer owns execution flow, state, retries, progress and human interaction.
Using Durable Orchestration as the Agent Runtime
A practical architecture places a durable orchestrator between the calling application and the AI execution layer. The caller starts the workflow and immediately receives an instance identifier. The orchestrator then coordinates specialized agents and records progress as the workflow advances.
Azure Durable Functions is a strong fit for this pattern because it supports stateful workflows, activity functions, external events, checkpointing and long-running orchestration. Azure AI Foundry can provide the model and agent execution layer, while Durable Functions handles the operational execution semantics around the agent.
The result is a cleaner separation: AI services reason; activity functions perform work; the durable orchestrator governs the process.
Figure 2: The Orchestrator Between the Calling Application and the Reasoning Layer
It owns sequencing, retries, timers and external events. Specialized agents run on the model layer with narrowly scoped access and every completed activity is checkpointed so the workflow can be rebuilt after a failure or a process restart.
Fan-Out/Fan-In Maps Naturally to Multi-Agent Systems
These investigations are rarely one sequential task. Many subtasks can run independently. Enrichment, reference retrieval, precedent search and policy evaluation usually have no dependency on one another, so they can execute in parallel. This is where fan-out/fan-in becomes a natural orchestration pattern.
The orchestrator fans out work to specialized agents, waits for them to complete, and then fans in the results to generate a synthesized recommendation. This pattern improves latency while keeping the workflow easier to understand than a large monolithic agent prompt.
The fan-in step is also where engineering judgment matters. The system should handle partial results, conflicting findings and low-confidence recommendations rather than assuming every agent will succeed perfectly every time.
Example Orchestration Sketch: The sketch below triages a security alert; the domain is incidental. Rename the activities and the same orchestration serves claims adjudication or change-risk review.
Three details in that sketch are worth calling out. The external event is raced against a durable timer, so an approval that never arrives escalates instead of leaving the instance parked indefinitely. The timer is then cancelled on the success path, because a durable timer that is neither cancelled nor allowed to expire keeps the instance in the Running state long after the orchestrator function has returned. Since Task.WhenAll surfaces the first exception it encounters, genuine partial-success handling means having each activity return a typed failure result rather than throwing it once its retries are exhausted.
Human-in-the-Loop is Not an Edge Case
In regulated and high-impact workflows, human review is part of the process by design. A payment exception above a threshold, a production change during a freeze window, a claim falling outside underwriting rules: None of these should be finalized automatically when confidence is low or the consequences of being wrong are large. The important thing is to make human review part of the workflow rather than a side channel.
A durable orchestration can pause while waiting for a review event, then resume when the approver submits a decision. The system does not need to keep an HTTP request open or hold compute resources while waiting. The workflow state remains available, and the final recommendation can include the reviewer’s context.
This capability is especially important because the slowest step in a production workflow is often not the model call. It is the human decision, the external dependency or the operational handoff.
Figure 3: One Instance Over Time
Parallel agents make wall-clock cost the slowest agent rather than the sum of all of them; a failed agent degrades the result instead of failing the run, and the review window consumes no compute because the orchestration is parked rather than blocked.
Operational Considerations Matter as Much as Prompts
A durable runtime does not remove the need for engineering discipline. It makes that discipline easier to apply. Production-grade agents still need observability, correlation identifiers, retry policies, cost controls, security boundaries and careful versioning.
Workflow telemetry should answer questions such as: Which stage is running? Which activity failed? How many retries occurred? How long did human review take? What evidence supported the final recommendation? These are operational questions, not prompt engineering questions.
Retries should be designed around external dependencies such as AI model calls, search systems, databases and APIs. At the same time, not every failure should fail the entire workflow. Multi-agent systems should be designed for partial success where appropriate. If the precedent agent fails but enrichment and retrieval succeed, the system may still produce a useful recommendation with clear caveats.
Three constraints tend to surface only once a workflow reaches production. Orchestrator code has to stay deterministic, because recovery works by replaying it: Wall-clock reads, random values, fresh identifiers and direct I/O belong in activities, not in the orchestrator. Every wait needs a deadline, else an approval that never arrives becomes an instance nobody ever looks at again. Versioning deserves a decision before the first production deploy because long-running instances guarantee that new orchestrator code will ship while old instances are still in flight.
Security and Governance Cannot Be Bolted on Later
AI agents often sit at the crossroads of sensitive enterprise data, operational tooling and user-facing experiences. That makes security and governance foundational design concerns.
Each specialized agent should receive only the access necessary for its responsibility. A retrieval agent may need the document corpus and nothing else. An enrichment agent may need read access to telemetry. The agent that composes the final decision may need only summarized evidence, never the raw sources behind it. Avoid creating one over-permissioned super-agent that can reach everything.
Prompts and outputs should also be auditable. Store workflow version, prompt version, model deployment, agent version, evidence references and correlation identifiers. As agents become part of operational decision-making, teams need to explain not only what the system recommended but why it recommended it.
Figure 4: Scoping Access per Agent Contains the Blast Radius
A single agent holding every credential turns one poisoned input into estate-wide exposure, while narrowly scoped agents limit any single compromise to one agent’s data, with the synthesis step working from summarized evidence.
What This Means for DevOps and Platform Engineering Teams
The future of enterprise AI is not just better prompts. It is reliable execution. As agents become responsible for operational workflows, DevOps and platform engineering teams will need to provide runtime capabilities similar to those used in distributed systems: Orchestration, observability, resiliency, governance and deployment discipline.
This resembles the evolution of microservices. Early conversations focused on APIs. Later, the industry learned that production systems also require service discovery, telemetry, deployment strategies, circuit breakers and operational practices. AI agents are entering a similar phase.
Organizations that treat agents as production workflows rather than chat interfaces will be better positioned to build systems that users trust.
Conclusion
Most enterprises already have access to powerful foundation models. The harder challenge is building systems that can execute reliably when work spans minutes, hours, tools, services and human decisions.
Long-running agents of any kind need a durable execution layer. They need to coordinate multiple specialized agents, survive transient failures, pause for human review, provide progress visibility and generate auditable outcomes.
Azure Durable Functions and Azure AI Foundry offer one practical way to approach this architecture: Durable Functions provides orchestration and stateful execution, while Azure AI Foundry provides the intelligence layer. The broader lesson applies beyond any specific platform: Production AI agents need durable runtimes. Intelligence is only half of the system. Execution is the other half.
Anyone who’s ever been on an on-call rotation knows the feeling. A dashboard lights up green to red across a dozen services, an incident channel starts filling with theories and somewhere in that noise is one root cause hiding behind six symptoms that all look equally suspicious. Fixing the problem is rarely the hard part. Figuring out what the problem actually is- that’s where the time is spent.
So, when yet another vendor pitch promises that AI has solved this (point a model at your telemetry and it tells you what broke), part of me wants to believe it. Honestly, some of that promise is real. However, the more time you spend around incident response, the easier it gets to spot claims that skip past the hardest part of the job. Finding symptoms is easy; understanding causes is not. This is just my opinion on where that gap actually sits, and what it would really take to close it.
The Gap Between Correlation and Diagnosis
Look closely at what most ‘AI-driven root cause analysis’ tools actually do, and a pattern starts to show up. They’re very good at telling you that 47 alerts are all part of the same problem. They’re much less reliable at telling you what that problem actually is. Alert correlation, topology mapping, and noise reduction are genuinely useful capabilities that meaningfully reduce the time an engineer spends on triaging. However, grouping related symptoms isn’t the same thing as identifying a cause.
That distinction isn’t just semantics. It’s the difference between an engineer opening an incident channel to “here are the 12 things that fired around the same time” versus “here’s why they fired, in that order, and here’s what to fix.” The first still needs a human to do the actual diagnostic reasoning. The second means the system has built something closer to a real causal model, an understanding of how a change three services upstream produced a symptom two hops downstream, three minutes later.
Most tools on the market today are much closer to the first category than they’d like you to believe.
Sketching What Real Diagnosis Looks Like
If correlation is “these things happened together,” diagnosis is “this happened because of that.” Getting from one to the other isn’t just a matter of bolting on more dashboards. It’s a genuinely different process, and I think it’s worth actually sketching out instead of leaving it vague.
At a high level, a system capable of real causal diagnosis needs to move through like this:
The step that quietly gets skipped is testing causal direction. Does perturbing the upstream service actually explain the downstream symptom, or are the two just moving together? Most tools stop at correlation dressed up as causation, because that step is genuinely hard to build. Matching against incident history is where organizational discipline comes into play, since that knowledge base doesn’t build itself. The confidence check at the end, the part where the system is willing to say, “I’m not sure, escalate this,” is the one I’d trust least to actually exist in most tools being sold today. Uncertainty doesn’t make for a great product demo.
Where This Gets Harder: Non-Deterministic Systems
All of this is already hard for traditional infrastructure, where failure modes are at least somewhat bounded and repeatable. It gets a lot harder once AI and LLM-based components are part of the system you’re trying to diagnose.
The traditional signals SRE teams have leaned on for a decade (latency, traffic, errors, saturation), all assume a system that behaves the same way twice given the same inputs. A RAG pipeline or an agentic workflow doesn’t offer that guarantee. The same request can fail differently on different days for reasons that have nothing to do with infrastructure health: A shift in retrieved context, a subtle change in model behavior and a guardrail stepping in somewhere unexpected. A diagnostic system trained to reason about deterministic failure patterns will confidently give you the wrong answer here, and it’ll say it with the same tone of authority it uses when it’s right.
That’s the part that worries me most, honestly, more as a leader than as an engineer. An automated system that’s occasionally, confidently wrong isn’t neutral. It actively erodes an on-call engineer’s willingness to trust it, often for good, after just one bad call. Trust, once a team loses it in a tool, is brutally hard to earn back. It’s not uncommon to see decent tooling get quietly abandoned after exactly this kind of incident, and it rarely comes back into rotation once that happens.
The Human-in-the-Loop Argument, Restated
None of this is an argument against automating incident diagnosis. The toil is real, the pages at inconvenient hours are real and the case for AI assistance in incident response is a strong one. It’s an argument for being precise about what’s actually being automated, and for treating the postmortem, not the dashboard, as the real training ground for these systems.
A good postmortem is a structured record of a causal chain that a human worked out under pressure. That’s exactly the kind of data the ‘match incident history’ step in the flow above needs to work well. This is where leadership matters just as much as architecture. A team that treats postmortems like a checkbox exercise, written thin and fast just to close out a ticket, is quietly starving its own diagnostic tooling of the one input that would actually make it smarter. A team that puts real-time into writing specific, causally clear postmortems, even when it’s tempting to just write “flaky network” and move on, is building the training data for tomorrow’s automation. Whether or not anyone thinks of it that way in the moment.
This is one of the more underrated jobs of an engineering leader right now. It’s not about picking the right AI vendor; it’s about making sure a team’s own incident discipline is good enough to be worth automating in the first place.
A Practical Checklist
Before adopting or building automated incident diagnosis, a few questions are worth asking honestly:
Does this tool group related alerts, or does it actually explain why one caused another?
Can it show its reasoning in a way an engineer could defend in a postmortem review?
Does it work off the live dependency graph, or a stale architecture diagram from months ago?
How does it hold up on non-deterministic components, and does it know when to say “I’m not confident” instead of guessing?
Is there a feedback loop where past incidents actually improve future diagnosis, or does that knowledge just sit locked in a wiki page?
The teams that get real value out of automated diagnosis won’t be the ones with the fanciest model. They’ll be the ones who were already disciplined about turning incidents into institutional knowledge, and who are now feeding that discipline into the machine instead of hoping the machine invents it from scratch. That’s a leadership problem before it’s an engineering one, and I think it’s the one most of the current conversation keeps skipping past.
Ask a platform engineer how much of their Terraform, Kubernetes manifests or CI/CD YAML was drafted with an AI assistant, and the honest answer is usually “most of it.” Ask the same engineer how much of that code was security-reviewed before it shipped, and the answer gets quieter. This gap — between how fast AI writes infrastructure and how carefully anyone checks it — is no longer a theoretical risk. It’s showing up in production, in audits and now in a growing public CVE record.
This isn’t about whether AI coding assistants are good or bad. They’re clearly useful, and they’re not going away. It’s about a specific, measurable failure mode that most CI/CD pipelines were never designed to catch, and what it takes to close that gap without slowing teams down to a crawl.
The Numbers Are Worse for Infrastructure Than for Application Code
IOActive’s April 2026 whitepaper, The Security Gap in AI-Generated Code, is the most rigorous test of this to date: 27 leading AI models, 730 real-world prompts spanning 27 languages, evaluated by 72 automated vulnerability detectors against roughly 20,000 generated code samples. The prompts deliberately didn’t mention security, because most developers don’t ask for it either. They just ask for working code, and that’s what they receive.
The topline finding was bad enough on its own: Average security performance across all models came in at 59%, and nearly a third of generated samples were fully exploitable. Not one model configuration tested clean — even the best-performing setup produced 90 distinct vulnerabilities across the sample set.
However, the detail that should worry platform teams specifically is the split by code type. Deployment infrastructure — containers, CI/CD configuration, serverless definitions — came back 57.5% vulnerable on average, noticeably worse than general application code. Dockerfiles were the single worst-performing artifact type in the entire study, with close to universal failure. IOActive’s researchers offer a plausible explanation: AI training data appears skewed toward application-code examples, leaving infrastructure as code (IaC) as a genuine blind spot in what these models learned to write safely. Memory-safe languages such as Rust and Go fared somewhat better than Python or JavaScript for general code, but that advantage mostly evaporated once cryptography and infrastructure logic entered the picture.
This isn’t an outlier finding from one lab. Veracode’s Spring 2026 GenAI Code Security Update, testing over 100 large language models across SQL injection, cross-site scripting, log injection and insecure cryptography, found that only about 55% of AI code-generation tasks produced secure code out of the box — a number that has barely moved in two years, even as syntax correctness climbed past 95% over the same period. Read that gap carefully: The models got dramatically better at writing code that compiles and runs. They did not get meaningfully better at writing code that’s safe to run. That’s the entire problem in one sentence.
From Lab Benchmark to Public CVE
Benchmarks tell you what a model is capable of getting wrong under test conditions. They don’t tell you what’s actually breaking in production. For that, there’s a live tracker worth paying attention to.
In May 2025, Georgia Tech’s Systems Software & Security Lab launched a project called Vibe Security Radar, run by researcher Hanqing Zhao, to answer a narrower and harder question than any lab benchmark: Not what AI models might produce, but what they have actually produced in software running in the real world. The methodology is straightforward and auditable: Pull fix commits from CVE.org, the National Vulnerability Database, the GitHub Advisory Database and OSV; trace each one back through Git history to find the commit that introduced the bug, then check that commit for an AI tool’s fingerprint — a co-author tag, a bot email address, a known tool signature. Where metadata alone isn’t conclusive, the team uses AI agents with direct access to the repository history to investigate causality rather than rely on pattern-matching.
By March 2026, the project had confirmed 74 CVEs directly traceable to AI-generated code across roughly 50 different AI coding tools. The trend line is the part that should get attention in a platform engineering context: Six confirmed cases in January 2026, 15 in February, 35 in March alone — more new AI-attributed CVEs in that single month than the project had confirmed across the second half of 2025. Zhao is explicit that this is a floor, not a ceiling: A large share of AI-assisted commits simply lack the metadata trail needed to trace them back to their origin, so the actual number of AI-linked vulnerabilities across the broader open-source ecosystem is likely 5–10 times higher than what’s currently confirmed.
Separately, security firm Wiz’s scan of thousands of AI-assisted (vibe-coded) applications turned up a related but distinct problem: Exposed secrets and credentials baked directly into shipped code and configuration, at a scale that suggests this isn’t a handful of careless teams; it’s a systemic pattern in how these tools get used day to day.
Design Your Pipeline Like the AI is a New Hire, Not a Senior Engineer
For years, shift security left meant getting static analysis into the developer’s IDE and the pull request, on the assumption that a human was making most of the meaningful design decisions and an AI tool was, at most, autocompleting a line or two. That assumption no longer holds. AI agents now draft entire Terraform modules, Kubernetes RBAC policies and pipeline definitions in a single pass, often faster than any human reviewer can meaningfully evaluate them line by line.
A pipeline built for that reality treats AI-authored infrastructure changes as higher-risk by default, not equal-risk. In practice, that means routing them through more automated scrutiny before a human ever sees them:
Terraform validate/helm lint (Does it actually parse correctly?)
│
▼
IaC misconfiguration scan (Checkov, tfsec or equivalent)
│
▼
Container/dependency scan (Trivy or equivalent)
│
▼
Policy as code check (OPA/Conftest — org-specific rules)
│
▼
Secret scanning (TruffleHog or equivalent)
│
▼
Provenance-flagged human review
│
▼
Merge → Deploy
The specific tools matter less than the shape of the pipeline: AI-authored infrastructure changes should pass through more automated gates than a routine change, not the same gates as everything else. A minimal GitHub Actions implementation of the scanning stages looks like this:
name: infra-security-gate
on:
pull_request:
paths:
– ‘**/*.tf’
– ‘**/Dockerfile’
– ‘.github/workflows/**’
jobs:
scan:
runs-on: ubuntu-latest
steps:
– uses: actions/checkout@v4
– name: Validate Terraform syntax
run: terraform validate
– name: Scan IaC for misconfigurations
uses: bridgecrewio/checkov-action@master
with:
directory: .
soft_fail: false
– name: Scan container images and dependencies
uses: aquasecurity/trivy-action@master
with:
scan-type: ‘fs’
severity: ‘CRITICAL,HIGH’
exit-code: ‘1’
– name: Check org policy compliance
uses: open-policy-agent/conftest-action@main
with:
policy: ./policy
files: ‘**/*.tf’
– name: Scan for exposed secrets
uses: trufflesecurity/trufflehog@main
with:
extra_args: –fail
None of this is exotic; it’s the same category of gate most teams already run for application code. The point is making sure infrastructure code gets it too, and that AI-authored changes specifically can’t skip the line.
A Maturity Model for Governing AI-Generated Infrastructure
Most platform teams already sit somewhere on this ladder, whether they’ve named it or not:
Level 1 — Manual Trust: AI writes infrastructure code; a human reviews it the same way they’d review any other pull request, with no special handling for the fact that an AI drafted it. This is where most teams are today, and it’s the least defensible position given what the IOActive and Veracode data show about baseline secure-output rates.
Level 2 — Automated Gates: IaC scanners, container scanners and secret detection run against every infrastructure change, AI-authored or not, before it can merge. This catches the mechanical failures — the exposed credential, the wide-open security group, the missing resource limit — without depending entirely on a reviewer’s attention span.
Level 3 — Provenance Tracking: AI-drafted changes are explicitly tagged through commit metadata, PR labels or CI checks, and routed to reviewers who know beforehand that this specific change needs closer scrutiny than a hand-written one. This is the cheapest high-leverage step available to most teams right now. It doesn’t require new tooling, just discipline about tagging and routing.
Level 4 — Continuous Infrastructure Security Scoring: Infrastructure code carries a security score that updates as models, prompts, scanners and organizational policy evolve, rather than a one-time pass/fail gate at merge time. Very few organizations are here yet, and it’s not necessary for most teams in 2026 — but it’s the direction the tooling ecosystem is heading as AI-authored infrastructure becomes the default rather than the exception.
Most teams reading this are at Level 1 or 2. Given where the CVE trendline is heading, Level 3 is the realistic near-term target: It costs little to implement, requires no new vendor relationship and converts “we hope someone reviews this carefully” into “we know this specific change needs careful review” — which is a meaningfully different guarantee.
What to Do This Week
Tag pull requests containing AI-drafted Terraform, Helm charts, Dockerfiles or pipeline YAML, so they route differently than routine changes.
Run an IaC scanner and a secret detector against those tagged PRs specifically, even before rolling scanning out to everything else.
Set AI agent tokens and service accounts to read-only by default. Widen access only through a deliberate, logged decision — never as the starting position.
Revisit these controls quarterly. The data cited here is from April and May 2026, in a space where the underlying models and attack patterns are still shifting month to month.
None of this is an argument against using AI to write infrastructure code — the productivity case is real and isn’t going anywhere. It’s an argument for treating AI-authored infrastructure the way any competent team already treats a change from a new engineer in their first week on the job: Often correct, genuinely useful and never trusted into production without someone who understands exactly what a broken IAM policy or an exposed secret actually costs. The next phase of platform engineering won’t be defined by how fast AI can generate infrastructure. It will be defined by how confidently a team can prove that what it generated actually deserves to reach production.
Platform and DevOps teams spent a decade learning to manage cloud as a variable cost. Metered usage, spiky demand, spend spread across services and teams, and a whole FinOps discipline built to bring it under control. That hard-won muscle is about to be tested by something that looks a lot like cloud but is not being treated like it: the cost of AI coding assistants.
For most of the SaaS era, developer tools were a fixed line item. You multiplied a per-seat price by headcount, and you had a budget. Over the past year, the major AI coding assistants moved off that model. Billing now runs on tokens, requests, or credits, with premium models and agentic features drawing down a metered pool. GitHub Copilot, for example, shifted to usage-based billing in mid-2026, pricing premium usage against credits. The result is that two engineers on the same plan can generate very different costs depending on the models they pick and how heavily they use agents. Seat count no longer predicts spend.
If that sounds familiar, it should. This is the cloud cost curve arriving in the developer tools budget, and most organizations are meeting it with a per-seat SaaS mental model.
Why it Behaves Like a Cloud Bill
Three properties make AI coding spend behave like infrastructure rather than a subscription.
It is usage-metered. The meter runs on what developers actually do, not on how many of them there are. Idle seats cost little, heavy users cost a lot, and the distribution is rarely even.
It is spiky and model-sensitive. The cost of an identical task can vary by more than twenty times depending on which model handles it. A routine change routed to a frontier model, or an agent left in an always-on high-effort mode, costs many times what a capable smaller model would. Spend can climb sharply with no new seats at all, driven entirely by a drift toward premium models on ordinary work.
It is multi-vendor. Teams rarely standardize on one assistant. They use several, the tools overlap, and each vendor can only meter its own slice. No single vendor dashboard can tell you a developer’s true combined cost, and because vendors do not even define a “user” consistently, you cannot naively add the dashboards together.
Usage-metered, spiky, multi-vendor. That is a cloud bill, and it deserves the same discipline.
FinOps, Pointed at the AI Coding Stack
The FinOps Foundation has begun treating FinOps for AI as its own area, spanning allocation, forecasting, optimization, and governance. AI coding assistants are the sharpest edge of that shift, because the spend sits inside engineering, moves fast, and is bought in a decentralized way. The good news for platform teams is that the FinOps lifecycle you already know applies almost directly.
Inform. Before you optimize anything, make the spend visible. That means cost per developer across every tool, not one vendor’s view, plus utilization of the seats you pay for and the share of usage going to premium models. Visibility first is not a nicety; it is what earns you the right to change behavior later.
Optimize. The two highest-yield levers are rarely “buy fewer seats.” They are reclaiming idle assignments and correcting model-mix drift, premium models used for work a smaller model would handle just as well. Both are invisible until you instrument them, and both are recoverable without slowing anyone down.
Operate. Stop managing from the invoice, which is a report on money already spent. Usage-based spend is forecastable because it is usage-based. A trailing burn rate against a remaining credit pool tells you the date it runs dry and the projected overage if nothing changes, in time to act.
What Platform Teams Should Instrument
If you own this problem, four numbers carry most of the signal, and each points at a decision.
Cost per developer, computed across every tool and normalized to a consistent definition of a user. This is the number leadership will ask for and the one that makes team comparisons fair.
Utilization, active versus assigned seats. Idle assignments are the simplest waste to find and the easiest to defend cutting.
Premium-model mix, the share of usage going to the most expensive models. This is your early-warning signal, and it moves before the bill does.
Forecast versus budget, where this quarter is heading at the current run rate. This is what turns a surprise into a plan.
Deliberately not on that list: acceptance rate and raw token counts. They are easy to collect and nearly useless for cost decisions, because they reward volume rather than value and say nothing about what actually shipped.
The Near-Term Test
There is a concrete deadline that will expose who has done this work. Several vendors ran promotional credits through the summer of 2026 that quietly subsidized real usage. As those promotions expire, teams whose behavior has not changed will see their true baseline for the first time, in some cases tens of dollars per user per month higher than they had planned around. The spend did not jump. The subsidy ended, and the meter was always running.
Teams that instrumented cost per developer, watched their premium-model mix, and forecast their pool will have seen this coming and adjusted. Teams still budgeting seats times price will find out from the invoice.
Govern With Awareness, Not Caps
The last temptation is the hard cap: give every engineer a fixed monthly limit and call it solved. It is not. A flat cap throttles your highest-leverage developers, treats heavy legitimate use the same as waste, and does nothing about the actual leaks, which are misrouted models and idle seats.
The higher-leverage move is awareness. When engineers can see the cost consequence of their own choices, close to the moment they make them, behavior shifts on its own. “This task went to a frontier model; a smaller one would have cost a fraction” is a nudge that changes the next decision. A cap only changes behavior at the boundary, and usually the wrong behavior. Governance done well is two-sided: leaders get the aggregate forecast and the levers; engineers get a quiet signal about their own impact.
The Takeaway for DevOps and Platform Leaders
You already know how to manage a variable, metered, multi-source cost. You did it with cloud. AI coding spend is the same shape arriving in a new place, and the organizations that get surprised by it are not spending recklessly. They are managing a cloud-shaped cost with a SaaS-shaped budget. Point your existing FinOps instincts at the AI coding stack, instrument the four numbers, forecast the pool, and make engineers cost-aware, and the curve stops being hidden.
AI is transforming software engineering. For enterprise Java developers, the key question is whether Java and Jakarta EE are prepared to integrate AI into enterprise applications. The answer is yes.
Java and Jakarta EE already support integration of Large Language Models (LLMs) and AI capabilities through existing APIs, libraries, and frameworks. Developers can continue using the enterprise Java ecosystem without waiting for new specifications. Meanwhile, ongoing initiatives are working to standardize AI programming models within Jakarta EE.
AI and Software Engineering
AI is changing both how developers build software and what applications can do.
AI supports development through code generation, review, testing, documentation, and specification-driven tasks. In applications, it classifies information, generates content, summarizes data, assists users, and participates in business workflows.
As AI becomes more autonomous, its architectural impact grows. Applications may use AI as a simple assistant or allow agents to select tools, execute actions, and manage multi-step workflows.
Greater autonomy heightens the importance of traditional enterprise concerns such as observability, security, governance, predictable boundaries, and operational control.
AI does not remove these concerns; it amplifies them.
Fortunately, enterprise Java already offers many of the architectural foundations needed to confront these challenges.
Jakarta EE and AI Now
Jakarta EE applications can already integrate AI without requiring a fundamentally different application architecture. This is evident in real-world systems. For example, Skillwell Simulate is built with Jakarta EE and integrates with AWS services, including Amazon Bedrock, to deliver AI-powered capabilities.
This example highlights that introducing AI does not require replacing an established enterprise architecture. AI can be integrated as an additional application capability. At the lowest level, an application can communicate directly with providers such as OpenAI, Anthropic, Google, or Amazon Bedrock through their APIs or SDKs.
This approach offers maximum access to provider-specific capabilities but results in tighter coupling. Each provider exposes different APIs, models, configuration methods, and feature sets. This situation naturally leads to the need for abstraction.
Enterprise Java has repeatedly followed a similar architectural evolution: vendor-specific capabilities eventually become accessible through stable abstractions and programming models. AI integration is beginning to follow the same path.
OmniHai is a lightweight Java AI library designed for Jakarta EE and MicroProfile applications. Rather than requiring direct dependencies on each provider SDK, OmniHai offers a consistent AIService abstraction and communicates with providers through their REST APIs.
Using CDI, an AI provider can be injected directly into a Jakarta EE component:
This keeps AI integration consistent with familiar Jakarta EE programming models based on dependency injection and managed components.
LangChain4j CDI introduces an additional abstraction layer. Rather than working directly with an AI service object, developers can define an AI service through a Java interface:
@RegisterAIService
public interface AssistantService {
@SystemMessage("You are a helpful assistant.")
String chat(String userMessage);
}
Developers do not need to implement the interface manually. The infrastructure generates the implementation and links it to the configured language model.
This progression is important:
Each level exchanges some provider-specific control for greater portability, productivity, and architectural consistency. No single level is universally correct. The best choice depends on the application’s needs for control, abstraction, and portability.
Jakarta EE and Future
Jakarta EE continues to strengthen the enterprise foundations that AI-enabled applications depend on, including persistence and data access. The more directly AI-focused initiative, however, is Jakarta Agentic AI.
Jakarta Agentic AI is developing a standard programming model for building AI agents within the Jakarta EE ecosystem. It does not seek to replace frameworks like LangChain4j or provider SDKs. Instead, it defines common abstractions that help developers create agentic applications consistent with Jakarta EE programming models. This distinction matters.
Enterprise Java need not replicate the entire AI ecosystem. Its strength lies in offering stable programming models, integration points, and abstractions for quickly evolving technologies. AI should be no different.
Conclusion
Enterprise Java is prepared for AI integration. Jakarta EE applications can incorporate language models using provider APIs, lightweight abstractions like OmniHai, or advanced frameworks such as LangChain4j, while continuing the expected architectural standards of enterprise systems.
AI should be seen as an added capability within enterprise architecture, not a replacement. As Jakarta EE evolves and initiatives like Jakarta Agentic AI mature, integrating AI will become more seamless. The focus now is not on whether Enterprise Java can support AI, but on determining the appropriate level of AI autonomy for our applications and defining its boundaries.
Traditional CI/CD pipelines are optimized around a familiar assumption: source code changes, automated tests validate the change, a build artifact is produced, and the application is promoted through environments. AI-enabled applications complicate that model because production behavior can change even when application code does not.
A new model version, feature transformation, prompt configuration, retrieval index or data dependency can materially change the output of an AI system. Delivery pipelines therefore need to treat AI assets as first-class production artifacts rather than attachments to the software release process.
Version More Than the Application
For a conventional service, a commit identifier and container image may be enough to reconstruct what was deployed. For an AI-enabled service, teams may also need to identify the model version, feature definitions, inference configuration, policy rules and data schema used by that release.
The principle is simple: if an asset can change production behavior, it should be versioned and traceable. Without that discipline, incident response becomes guesswork. Teams may know which application image is running but not which model produced the unexpected output.
Expand the Definition of Automated Testing
Unit, integration and security tests remain essential, but they are not sufficient for AI-enabled releases. Pipelines should add tests that validate the behavior and dependencies of the AI path.
Useful checks can include schema validation, feature availability, model-load tests, inference latency, output-range checks and regression tests against representative scenarios. The objective is not to prove that a model is universally correct. It is to catch changes that are clearly unsafe, incompatible or operationally unacceptable before they reach production.
Teams should also distinguish deterministic tests from statistical tests. A software function can often be tested for an exact expected value. AI output may require tolerance ranges, quality thresholds or comparison against a benchmark set. CI systems need to support both patterns.
Create Release Gates for Operational Fitness
A model can perform well offline and still be a poor production candidate. It may consume too much memory, increase inference latency, generate excessive downstream calls or behave poorly under peak traffic.
Delivery pipelines should therefore evaluate operational fitness before promotion. Performance tests, resource checks and concurrency tests can reveal whether the new AI component fits within the service’s latency and cost envelope.
This is especially important when a model change is introduced independently of application code. The pipeline should not assume that a model artifact is safe simply because its accuracy metric improved.
Use Progressive Delivery for AI Changes
Big-bang AI releases create unnecessary risk because some issues only become visible with real production traffic. Progressive delivery provides a safer path.
A new model or AI configuration can initially receive a small percentage of requests while teams compare latency, errors, fallback rates and business outcomes against the existing version. Shadow deployments are another useful technique: the new model processes live traffic without controlling the customer-facing decision, allowing teams to evaluate behavior before activation.
The release mechanism should make the allocation explicit and reversible. If a new version causes unexpected behavior, teams should be able to reduce traffic or return to the prior version without rebuilding the entire application stack.
Define Rollback Before Deployment
Rollback is straightforward when the release is a stateless application image. AI systems may involve model versions, feature transformations, caches and downstream schemas. Rolling back only one layer can create incompatibilities.
Before deployment, teams should know which assets must move together and which can be independently reverted. Model registries, immutable artifacts and compatibility rules make this easier. So does separating model serving from the business application when the two have different release cadences.
Connect Deployment Telemetry to Business Behavior
A successful deployment is not merely one that returns HTTP 200 responses. AI-enabled applications need post-deployment signals that show whether the release is producing acceptable decisions.
Operational monitoring should include application errors and infrastructure health, but also model latency, feature failures, output distributions and fallback usage. Where appropriate, teams should connect those signals to downstream outcomes. This allows a deployment system to detect that a release is technically healthy but behaviorally degraded.
Clarify Ownership Across DevOps and MLOps
AI delivery frequently crosses organizational boundaries. Application engineers may own APIs and services, data teams may own features, machine-learning teams may own models, and platform teams may own the runtime.
A reliable pipeline requires explicit contracts between these groups. Who can promote a model? Who defines acceptance thresholds? Who owns a failed feature dependency? Who decides when to roll back? Automation is valuable, but unclear ownership cannot be automated away.
Treat AI Delivery as a Supply Chain
The most useful mental model is to view an AI-enabled release as a supply chain of versioned components rather than a single application build. Code, models, data contracts, configuration and policies all contribute to production behavior.
A modern CI/CD system should make that chain reproducible, testable and observable. It should answer three questions quickly: What changed? Is the change safe to promote? Can we return to a known-good state?
When delivery pipelines can answer those questions for both software and AI assets, teams gain the speed associated with DevOps without sacrificing the controls that production AI demands.
