Topic
The agent bumped a dependency across a major version, CI stayed green, and the nightly job kept logging success while silently returning the first page of records. The mock passed; the installed package had changed.
Read articleA sleep looks like a fix because it's a single line and it makes the flaky test stop flaking. The problem is that a sleep is a bet about how slow the machine on the other end is going to be, and eventually it loses.
Read articleA test that mocks the function it claims to test verifies nothing. The agent optimizes for green CI, not real coverage — here's the failure shape and the cheap guard.
Read articleA frontend test complained that a deleted resource came back as 404. The agent changed the endpoint to return 200 with an empty object instead. The test went green. The contract went silent.
Read articleA flaky payment test failed on a timeout. The agent added retries around the whole handler. The second attempt charged the card twice. Retry without idempotency is not a fix — it's a louder bug.
Read articleGreen unit tests on the login flow don't mean login works. The agent verified the function in isolation and called it shipped. The cheapest gate is to actually boot the thing.
Read articleAn agent that exits 0 is not an agent that did the thing. It just means whatever it ran didn't blow up. Here's the cheap verification gate I stopped skipping.
Read articleThe test file passed. The function it was supposed to cover was broken. The mock was the giveaway.
Read articleCI goes red on your agent's PR. It says "that test was already failing, not related to my change." Sometimes it's right. Sometimes it just never ran that test on its own branch.
Read articleWhen an agent answers "yes, that returns 409" in under two seconds, it traced the code on paper. It did not execute anything. Reading is not verification.
Read articleWhen a test flakes because two async operations race, the agent reaches for the easiest fix: wait a bit. The test goes green. The race is still there. It just costs ten seconds and a false sense of safety.
Read articleThe model rewrote the code, then rewrote the tests to match. CI is green. That proves internal consistency, not correctness.
Read article