The agent fetched the access token once at boot and cached it forever, so rotation broke the app hours later
We asked the agent to wire an outbound call to a third-party billing API. It read the auth guide, obtained a client-credentials token, and added a module-level constant: fetch the token once at startup, store it, reuse it for every request. The tests passed against a stub that always returned 200. The deploy went fine.
Around an hour later — the token's typical lifetime — production calls started returning 401, and restarting the process fixed it until the next rotation. The agent had modeled a short-lived token as a static credential, like an API key that never changes.
Why the happy path wins
The SDK guide shows one token exchange and one request. The agent copies that sequence: exchange credentials, hold the result, attach it to the next call. Nothing in the example mentions an expiry timestamp, a refresh, or a retry on 401, so nothing in the implementation appears.
- The tests stub the auth layer. A fake provider returns a valid token forever, so expiry never reaches the suite. Green proves the call shape, not the credential's lifetime.
- A long-running process is foreign to a short session. The agent runs against a script that exits; "fetch once" reads as a tidy optimization rather than a correctness bug.
- Nothing fails at boot. A clean local run and green CI confirm a design that only breaks after uptime, which the agent never observes.
The result is code that works for the first hour of every process and degrades silently after that. A restart masks it, which makes the failure look flaky rather than structural.
Store the token with a lifetime
A token is a value that expires; the secret used to obtain it is what rotates. Keep that distinction and model the expiry instead of ignoring it.
- Cache the token, not the exchange. Fetch lazily on first use rather than at import, store the returned expiry, and refresh a short window before it expires instead of waiting for a 401.
- Refresh on 401 once. When a token expires early or a clock drifts, retry the exchange once and replay the request; do not let a single expired token surface as a hard failure to the caller.
- Test the expiry path. Force an expired token in the suite and assert the provider refreshes and the request still succeeds. A test that only feeds a fresh token never exercises the branch that matters in production.
- Keep one token provider. Put exchange, expiry, and refresh behind a single provider instead of re-implementing the fetch at each call site; scattered copies drift the fastest.
- Rotate the secret, not the token. The client secret is the long-lived credential; the token it buys is short-lived and should be disposable.
Fetched once is not cached forever
Green after a clean deploy proves the request works against a stub, not that the credential survives an hour of uptime. When the agent holds a token for the process lifetime, the first failure lands after rotation, disguised as a flaky 401 that a restart hides. Model the expiry, refresh before it, and test the branch the happy path never reaches.