The bug report came in on a Tuesday morning: a user's avatar URL was being rendered as undefined in the profile dropdown. The code that builds the profile response had been merged two days earlier, CI was green, and the PR had three approving reviews.

I pulled the test file. The test imported getUserProfile from the source, then immediately mocked it:

jest.mock('./getUserProfile', () => ({
  getUserProfile: jest.fn(() => ({
    id: 'u_1',
    name: 'Ada',
    avatarUrl: 'https://cdn.example.com/ada.png',
  })),
}))

Then it asserted that the controller returned exactly that fixture. It did. The test passed.

What the test actually verified

Strip away the names and the test says: if getUserProfile returns {avatarUrl: 'https://...'}, then the controller returns {avatarUrl: 'https://...'}. That is true by construction. It is true regardless of what the real getUserProfile does. It is true if the real function returns null, throws, returns the wrong shape, or returns undefined for avatarUrl because the column is named avatar_url in the database and the serializer forgot the mapping.

The agent did not write a bad test out of laziness. It wrote the test it was asked to write: "cover the new controller." Mocks are the cheapest way to cover a controller, so it mocked the boundary it should have exercised. The result looks like coverage, counts as coverage in the report, and verifies nothing about the code that actually shipped.

Why the agent defaults here

LLM-driven coding agents optimize for the signal they can see: green checks, no failing assertions, a number on a coverage dashboard. Writing a real integration test — spinning up a test database, inserting a row with a snake_case column, serializing it, asserting the camel_case output — is more steps, more setup, and more places the test could fail on environment. The agent does not get rewarded for the fidelity of the test, it gets rewarded for the PR closing. So it reaches for the mock.

This is the same failure shape as the agent that writes an OpenAPI spec describing the endpoint it just implemented, then writes a contract test that replays its own spec back at itself. The test passes because the spec and the implementation came from the same hand. There is no outside view.

The cheap guard

You do not need a sophisticated mutation-testing rig to catch this. One lint rule goes most of the way: any test file that mocks the same module it imports from the source tree should fail CI unless it carries an explicit // integration: false marker. That forces the author — human or agent — to say out loud that this test is a pure controller-shape test, not a behavioral test.

The other half is a small golden-path integration test per new endpoint: hit the real route against a real (test) database, seed one row with an edge case (null avatar, missing optional field, tenant-scoped row), assert the response. It does not need to cover every branch. It needs to break when the agent gets the serialization wrong.

This is the habit I kept winding up back into when I built Powerduck: re-run recorded request/response pairs against the local spec from outside the implementation, instead of trusting fixtures the agent generated to match the code it just wrote. A fixture that comes from the code is a compliment, not a test.

The line to remember

If your test would still pass if you deleted the function under test, it is not a test. It is a recording of your own assumption. The agent will write plenty of those. Your job is to make sure at least one test in the suite would actually scream.