Category: Social – X

Writing code stopped being the constrained step. Agents take a task, read the repository, change files across a codebase, and iterate against the build and test feedback that already exists. What they cannot do is decide whether the result belongs in the system. That decision is where engineering expertise now sits, and it did not get easier when the typing did.

I spent years at Uber on the parts of the system that decide whether anyone ships, the monorepo and the build and the CI queue, and later on training models against that same codebase. Making it cheaper to produce a change moves the pressure downstream onto everything that has to certify the change is safe. That is most of what happened to our industry in the last two years, and it arrived faster than anyone had planned for.

Taste is Your Standards, Written Down

Taste in engineering is not preference. It is knowing what makes a change fit the system it lands in, which is a different question from whether it satisfies the request. An agent will satisfy the request, but it will not know about the architectural constraint someone argued about for three weeks, two years ago, or the convention the team enforces in review and never wrote down, or the dependency direction that is not supposed to reverse. Nobody put any of that in a prompt. It is in people’s heads.

An agent cannot read a person’s mind. So, the practical form of taste is writing the standard down where the code lives, as rules and review guidance under version control, sitting next to the service they govern. Kept there, the guidance gets reviewed like code, since it shows up in the same diff as the change it governs, in front of the person best placed to notice it has gone stale.

Volume works against you. Hand a model forty pages of standards and it applies them about as well as a new hire would. The guidance has to be cut down to what applies to the change in front of it. Standards you cannot pick from are standards the agent averages out.

Discernment is the Line Between Proposing and Deciding

The failure mode is code that compiles, reads fine, and is wrong for this particular system, which is not a thing a diff shows you. Sonar’s research found 61% of developers agree AI often produces code that looks correct but is not reliable, and 53% said it has already hurt their technical debt. Those numbers describe a review problem rather than a generation problem, and they are not going to be fixed by a better model.

The obvious failures are cheap. It is the expensive ones that get through, the change that solves the immediate problem and quietly leaves the system harder to test or operate, where every individual line reads fine and the damage only shows up if you already know how the pieces fit together. Catching that is discernment, and it comes from having run the system in production. Knowing the call paths and what the tests actually cover, which is never what the coverage number says. It’s knowing how the service behaves at 3 am.

We ended up building the same boundary into our own product, on the theory that a model is a bad place to keep a decision. The model proposes findings, the verdict is computed in code from the state of those findings rather than declared by the reviewer, and that one change removed an entire class of bugs where the reviewer would request changes on a pull request whose findings had all already been resolved.

Resolution works the same way, as a deterministic check against the diff, and the model does not get to un-resolve what the diff settled. When several reviewers run in parallel, the arbiter can only drop candidates, never add one. An arbiter having a bad day costs us a miss rather than an invention. The rule underneath all of it is to give the probabilistic layer the jobs where being wrong is survivable and keep the state machine deterministic.

Judgment Decides What Ships, and What is Allowed to Ship Itself

Almost nothing at this level is right against wrong. You are picking which failure you would rather have. Take the migration now and eat a week of dual writes, or defer it and let the schema rot another quarter, and the engineer who has carried the pager for that service knows things about how it fails that never make it into a prompt.

The judgment itself has not changed. Where it gets applied has. It used to be spent one pull request at a time, in review. Now most of it is spent once, in writing, on the conditions under which a machine may act without asking anybody.

Teams get there in a predictable order. Detection first, then remediation, then approval under criteria they wrote themselves, then merge. Nobody jumps to the last step, and what moves them along is evidence out of their own codebase rather than a benchmark.

Constrain by scope, not by instruction. Our agent cannot push a branch or merge a pull request on its own initiative, and that is blocked at the tool level rather than asked for in a prompt, since an instruction is a preference and a missing capability is a guarantee. Automatic approval can only approve, never request changes, and it fails open, so a broken judge costs a convenience instead of a merge. Scoping the objective matters as much as scoping the permissions, which is why a CI failure unrelated to the change goes down the retry path, instead of the fix path. “Make the test stop failing” is the objective you least want a capable agent pursuing.

The Bar Does Not Have to Drop

This part we can measure. Auto-approve and auto-merge leave us a cohort where our agent was the only reviewer on the pull request, and that share keeps climbing. Those merge much faster, which surprises nobody, and the changes in them run larger on average, which I did not expect and still cannot fully explain.

The part worth reporting is what did not move. Changes get requested at about the same rate whether the reviewer was a person or our agent. The agent blocks slightly more often than people do, not less. And developers dismiss its findings less often than they dismiss findings from each other. Anyone would have predicted the speed. What I did not expect was that taking the second reviewer out would leave the remaining one behaving much the way it had before, and that is the part I would want another team to try to reproduce.

It is correlational, and teams that switch autonomous merge “on” are not a random sample of teams. These are also in-review signals rather than post-merge ones, since reverts and incidents take longer to attribute. It is still the only comparison group I know of.

Make the Expertise Repeatable

Nobody reviews their way out of this by hand once generated code is most of the diff. Enforcement belongs on the layer that behaves the same way every time. Whether a merge gets blocked should not depend on which model happened to be cheap that week. Quality gates, tests, security scanning and policy checks decide. Contextual review takes the judgment calls, whether a change does what it claims and whether it fits the codebase, and hands them over as evidence for a person to weigh.

Auditability splits the same way. Identical reruns of a model are not a promise worth making, and an audit does not need them. What it needs is a record of what was reviewed, what was raised, and who decided. Every finding we produce is persisted and never deleted, only marked resolved or dismissed, so a pull request can be reconstructed a year later by someone who was not there.

Businesses own the software they release regardless of who typed it. Accepting generated code without reading it buys delivery now and pays for it later, with interest. The teams that get the most out of coding agents will be the ones that scale their standards as fast as they scale generation, and keep the decisions that need context with the people who are accountable for them.