The CI log shows a timeout. You send the failure to your agent. Two minutes later, it opens a PR: await new Promise(r => setTimeout(r, 500)) before the assertion. The test passes locally. You merge it.

Next week it flakes again, this time on a different machine, at a different hour. The sleep was 500ms. The race still takes 600ms on a loaded runner. Nobody deleted the sleep. Nobody remembered it was there.

The symptom, not the cause

A sleep() does one thing: it narrows the window in which the race can win. It doesn't close the window. On your laptop, the window might be 10ms wide. On a shared CI runner under load, it stretches. The agent picked a number that worked once and moved on.

This is the same shape as editing an assertion to match broken code. The test goes from red to green without the underlying problem moving. The difference is that a changed assertion usually gets caught in review. A sleep() looks like engineering.

What the fix should actually be

If the test depends on an async side effect — a database write, a dispatched event, a resolved promise — wait for that thing to be observable, not for a clock.

In Node, that means polling a predicate or using an event emitter, not setTimeout. In Python, it's await asyncio.wait_for(condition), not time.sleep. The rule is boring: wait on a signal, not a duration. If you can't observe the signal, that's a test design problem — the code doesn't expose what the test needs to know.

I've started asking agents to explain what state the assertion depends on before writing any fix. If they can't name it, they're going to reach for a sleep. When they can name it, the fix usually looks like a waitFor(() => expect(...)) or a callback hook, and the test stops flaking on every machine.

This happens in API workflows too

The same failure mode shows up when an agent is told to test an API endpoint that kicks off an async job. It sends the POST, then sleeps two seconds, then checks the GET. On a fast instance, the job is done in 200ms. On a cold start, it takes five seconds. The test passes sometimes.

The fix is the same: poll until the resource reaches the expected state, or use the webhook/callback the API already provides. A contract that says "the job completes asynchronously" should tell you what to wait for — a status field, a Location header, a retry-after. If the spec doesn't say, that's a spec gap, not a reason to guess a timeout.

We ran into this building Powerduck, where the spec is the source of truth for what an endpoint returns and when. An agent that has the spec in front of it can look up the async pattern instead of inventing a sleep. It's still the developer's job to reject the timeout fix, but at least the alternative is on the page.

The review trick

Next time you see sleep, wait, delay, or setTimeout in a test diff, ask one question: what is this waiting for? If the answer is "enough time," reject it. If the answer is a specific state or event, it's probably right.

How do you catch the sleep() fix in your codebase?