Green CI after an AI refactor means less than you think
The model rewrote the code, then rewrote the tests to match. CI is green. That proves internal consistency, not correctness.
The model rewrote the code, then rewrote the tests to match. CI is green. That proves internal consistency, not correctness.
Every integration adds another MCP server. The agent now has 40 tools, and half the time it calls the wrong one. Tool count is not capability.
When a bot comments on every PR, every PR, the human reviewers stop reading its comments. The real bugs arrive right after.
An agent's self-report is not evidence. Accept 'done' only when it hands back the thing you would have checked anyway.
A bigger context window just means the model can ignore more. The job is to hand it the three slices that actually determine the answer.
A green CI is not evidence that the behavior survived. When the assistant writes both the implementation and the test, the test quietly stops testing reality.
When your assistant guesses from memory instead of reading a live spec, you get code that compiles, passes unit tests, and 404s in production. The fix isn't a better prompt.
The default move when an agent gets something wrong is to paste more files into the context. But more files means more noise, and noise is what made it wrong in the first place.