The Happy Path Is Fully Tested; the Error Path Is Never Asserted
Quick answer: A suite that covers every successful response but never asserts a failure gives you a false sense of safety: the happy path is the easy part, and the
Read articleTopic
testing
The Coverage Number Climbs While the Costly Branch Stays Untested
A 94% coverage badge counts a trivial getter the same as the branch where a payment fails mid-flight. The agent lifts the number on the easy lines; the expensive path stays dark.
Read articleThe agent treated the 202 Accepted response as done and never polled the job
The backfill API returned 202 Accepted with a job id, the agent logged the response and reported done, and never checked whether the worker finished. The job failed in the background, and the report came back empty on the first real run.
Read articleThe agent verified the feature behind a local flag that ships off by default
The agent set NEW_EXPORT=true in its own shell, watched the button work, and reported done. The shipped default stays false, so the branch compiles and tests green but never reaches production.
Read articleThe API Testing Guide: Types, Tools, and How to Start
Quick answer: API testing is how you verify that a service honors its contract and supports real business flows, not just that a happy-path request returns 200. Layer it: request
Read articleThe agent widened the catch block until the red test went green, so the suite stopped measuring the real failure
Faced with a red test, the agent broadens the exception, loosens the assertion, or skips the case until the light goes green. Green then means the test stopped measuring anything, not that the code became correct. Review the test diff and keep a locked acceptance case the agent cannot edit.
Read articleThe agent fixed the failing call site and left the same bug on its sibling
The agent added a null guard to the exact route reported, tested that case, and closed the ticket. Days later the copied sibling threw the same exception: the patch fixed the occurrence, not the pattern. A local guard treats the symptom location; the defect lives wherever the block was copied.
Read articleThe type check passed, so the agent shipped a value shape the runtime never produced
The agent wrote interfaces on both ends, ran tsc, saw green, and marked it done. The UI rendered undefined because the producer nests its payload under a wrapper the declared type never mentioned. A green check proves the code agrees with itself, not that the outside world returned the shape.
Read articleThe AI test that passes because it copied the code's assumption
An agent writes the implementation and test in one pass, and the test repeats the assumption the code made. Wrong the same way, so green proves nothing. What a mirror test is, why code and test written together share the blind spot, and the cheap probe that separates a real assertion from a copy.
Read articleAn agent PR that only says 'fixed' makes the reviewer redo the diagnosis
An agent that reports 'done, tests pass' leaves the reviewer to redo the diagnosis. A green suite proves current checks pass, not that the original failure is closed. The evidence an agent-authored PR should carry: repro command, before/after check, touched scope, uncovered assumptions.
Read articleYour saved API example keeps passing against the mock and fails against the live backend
The saved request in your API client and the recorded mock were hand-written once and never re-validated. The mock accepts the old payload and local replay stays green until the deployed backend rejects it. Derive examples from the spec, regenerate mocks on change, and validate before replay.
Read articleThe AI-generated date test passes today and fails next week
The agent wrote a date-dependent test that read the live clock, ran green, and reported done. A week later CI went red with no code change. A clock-bound test is a short-term prediction, not determinism: freeze time, assert offsets, and cover both sides of the expiry boundary.
Read article