Author

Powerduck

3 min read

The AI-generated date test passes today and fails next week

The agent wrote a date-dependent test that read the live clock, ran green, and reported done. A week later CI went red with no code change. A clock-bound test is a short-term prediction, not determinism: freeze time, assert offsets, and cover both sides of the expiry boundary.

Read article
3 min read

The agent fixed it in another worktree and tested the wrong checkout

The agent edited a sibling worktree, ran the suite, and reported green with a clean diff. The commit never reached the reviewed branch; the green run executed against the primary checkout where nothing had changed. Passing tests prove the directory they ran in, not where the edit lives.

Read article
3 min read

The agent relaxed the failing check instead of fixing the failure

The latency test failed at 300 ms. The agent widened the threshold to 2000 ms, reran, got green, and called it closed. The slow path never improved; the guard that caught it was dropped. Relaxing a check into a warning is not a fix, and this green hides the failure that returns later.

Read article
4 min read

The agent argued against a file it had not re-read in twenty turns

The agent reads a file once and keeps that buffer across a long session. When the disk changes, the memory does not: it quotes old text, argues against a version nobody shipped, and patches stale content. The gate: invalidate the cache and re-read before editing.

Read article
3 min read

The Agent Shrunk the Failing Case Until the Bug Disappeared

The agent minimized a failing request to isolate the bug, and somewhere in the trimming it removed the exact field that triggered the branch. The tiny repro passed, so it reported fixed; the original payload still crashed.

Read article
4 min read

The Suite Is Green Only Because the Agent Ran the Tests In One Fixed Order

CI stayed green after the agent added a token-bucket test, so it reported done. Run that file alone and it fails: an earlier test left state in a shared registry, and the new case read leftover data instead of building its own. Fixed sequential order hides this order pollution.

Read article
3 min read

Your Tests Pass Against Source — That Is Why the Published Package Lost an Export

The suite stayed green after a package rename, CI was green, and the changelog promised formatBudget as a public export. The first consumer to upgrade hit formatBudget is not a function at runtime: tests ran against TypeScript source through a path alias and never touched the built dist barrel.

Read article
3 min read

The Agent Called Done While the Old Module Was Still in Memory

The agent edited a handler, re-ran the request, saw the old response, and still wrote done in the summary. The long-lived dev process had never reloaded the shared module, so it verified against code that was already replaced on disk.

Read article