Quick answer: An MCP tool is itself a client of the upstream API it wraps, and several model calls can funnel through one shared credential, so the tool should enforce its own outbound limit rather than rely on the upstream to absorb bursts. Read the upstream's 429, Retry-After, and RateLimit headers, smooth traffic with a token bucket or queue, cap concurrent calls, and when the upstream stays blocked return a structured error telling the model to stop and back off instead of hammering the endpoint.

Why the tool needs its own limit

A single MCP server can serve more than one host, and an agent can fire the same tool in a tight loop. Every one of those calls converges on the upstream account the server authenticates with. From the upstream's point of view, a burst of independent model decisions looks like one client exceeding its quota. If the tool passes every call straight through, the shared key gets throttled or temporarily banned, and every caller suffers at once. The boundary to protect is therefore inside the tool, between the incoming model requests and the outbound upstream request.

Read the upstream signals first

The upstream already states its limits. A 429 response may carry a Retry-After header in seconds or as a date, and RateLimit headers expose the limit, remaining count, and reset window. The handler should treat those as the authority on when to resume rather than guessing a fixed delay. Watching the remaining count lets the tool slow down before it hits the wall, not after. When no headers are present, fall back to exponential backoff with jitter so retries do not synchronize into another burst.

Controls on the server side

Control What it does
Token bucket Smooths bursts while keeping a steady average rate
Concurrency cap Bounds how many upstream calls run at once
Wait queue Lines up excess calls instead of rejecting them
Response cache Reuses identical reads to cut upstream traffic
Structured error Returns isError with guidance when blocked

The bucket and queue handle the common case where calls briefly exceed the steady rate, while the concurrency cap protects upstreams that limit simultaneous connections. Caching helps only for safe, repeatable reads; never cache a write or a per-user result across callers.

What the model should see

When the upstream remains blocked after the tool's own retries, the result should be a structured error, not a raw stack trace or silence. It should say the upstream is rate limited, give any reset time the server returned, and ask the model to wait or reduce the number of calls. A clear error breaks the loop in which the model interprets a failed call as a reason to retry immediately. The tool should also distinguish a 429 from an auth failure, since re-running a request with an expired token will not fix a quota problem.

Honest limits

Local testing can reproduce a 429, Retry-After, and the queuing behavior, but it cannot prove the upstream's real quota math, burst allowance, or how limits are shared across accounts or regions. A token bucket tuned against a mock may still mismatch the production policy, which can change without notice. To ground the settings, drive real requests against the upstream and observe the headers in a local workspace; Powerduck keeps each call and its response visible so the limits are calibrated on evidence rather than a guess.

Frequently asked questions

Is this the same as a gateway limiting tenants? No. This is outbound: the tool protects the shared upstream it calls. A gateway applies inbound limits to the callers reaching it.

Why not just forward every call? Independent model calls converge on one credential, so a burst can exhaust the upstream quota and block every caller.

What if the upstream sends no Retry-After? Fall back to exponential backoff with jitter and re-check the RateLimit headers on the next response.

Should the tool cache responses? Only safe, repeatable reads. Writes and per-user results must never be shared across callers.

The bottom line

Treat the upstream as a shared, finite resource: read its 429 and RateLimit signals, smooth calls with a bucket and queue, cap concurrency, and return a structured error that makes the model back off when the limit holds. For more on operating these servers, see the MCP server guide hub.