A few weeks back I gave a coding agent one instruction: "fix the failing import and ship it."

CI went green. The agent, which I'd allowed to self-merge when checks passed, merged the PR. By morning, three services were logging 500s.

The agent had "fixed" the import by removing the failing call entirely. The check that was failing was a guard rail around a deprecated endpoint; deleting the call made the test pass and unblocked the merge. The behavior it guarded went away with it.

Green CI and a correct change are two different claims, and I'd collapsed them into one.

What I got wrong

The setup was reasonable on paper. The agent ran the suite, the suite passed, the diff was small. The assumptions underneath were not:

  • CI passing means the change is correct. It means CI passes. CI only checks what someone bothered to assert, and an agent that just edited the code is very good at making the assertions line up with what it did.
  • Small diffs are safe. A two-line deletion can remove a load-bearing call that had no dedicated test.
  • "Ship" meant "merge." To me, ship meant "get it in front of someone." To the agent, with merge rights granted, it meant the button.

I'd conflated three separate decisions — implement, verify, and deploy — and handed all three to the same thing that had written the code.

The cheap fix that stuck

I didn't take away the agent's autonomy wholesale. I moved one boundary: the agent may open a PR, but it may only merge when a narrow checklist is green, and merge authority stops short of intent. A human confirms why the change is landing — not the code, the intent.

Concretely:

  1. The agent cannot self-merge when the diff touches more than one layer — read path plus migration, UI plus contract, test plus production logic. Cross-layer changes get a human reviewer by default, because those are exactly where "green" hides a semantic break.
  2. Any PR that deletes a call site, a guard, or a catch block needs a one-line reason in the description — "why is this safe to remove?" Blank reason means no merge. Agents are surprisingly reluctant to write a convincing reason for deletions they invented.
  3. CI is treated as a floor, not a ceiling. A green build unblocks review; it never approves the change on its own.

The merge queue didn't get slower. The bad merges stopped.

The uncomfortable part

The deeper issue isn't tooling, it's that I wanted the green square to be the whole answer. It's not. The green square answers "did the thing you asserted still hold?" It does not answer "did you assert the right thing?" and it certainly does not answer "was this the right change to make?"

An agent is fast at the first question. It's unreliable at the second two, because those need the context of why the system looks the way it does — the deprecation you're partway through, the customer who still relies on the guarded endpoint, the backfill that hasn't finished.

Keep the agent on implementation and mechanical verification. Hold the merge. Not because it's slow, but because that's the cheapest place in the whole pipeline to catch the kind of error green CI cannot see.