How to Fix 429 Rate Limit Errors in AI API Pipelines
6 min read · Updated Aug 5, 2026
A 429 error means you have exceeded a rate limit, but "rate limit" on most AI APIs is really three separate limits stacked under one status code: requests per minute, tokens per minute, and concurrent requests. Fixing it for good means figuring out which of the three you are actually hitting, the response headers usually say, then either queuing requests to stay under it, batching multiple items into fewer calls, or upgrading your usage tier. Not just wrapping every call in a retry loop and hoping.
Key takeaways
- A 429 rarely means one single limit. Most providers enforce requests-per-minute, tokens-per-minute, and concurrent-requests limits separately, and you can hit any of them independently.
- Check the response headers first (
x-ratelimit-remaining-requests,x-ratelimit-remaining-tokenson OpenAI) before guessing which limit you tripped. - A queue with a fixed concurrency cap prevents 429s proactively, instead of just reacting to them after they happen.
- Batching multiple small requests into fewer, larger calls reduces requests-per-minute pressure without reducing total volume processed.
- Retrying with backoff handles occasional spikes. It does not fix a pipeline that is structurally over its sustained rate limit, that needs an architecture change or a tier upgrade.
Rate limits are usually three limits wearing one name
OpenAI, Anthropic, and most other AI providers cap usage along at least three dimensions at once: how many requests you send per minute (RPM), how many tokens you process per minute (TPM), and how many requests can be in flight at the same time (concurrency). A pipeline can have plenty of headroom on requests per minute while completely exhausting its tokens-per-minute allowance with a handful of long-context calls, and both look identical from the outside: a 429 response with no further detail unless you read the headers.
Read the response headers before you guess
Every 429 response carries the exact numbers you need to diagnose it. Stop guessing which limit you hit and read them directly.
x-ratelimit-limit-requests: 3500
x-ratelimit-remaining-requests: 3499
x-ratelimit-limit-tokens: 90000
x-ratelimit-remaining-tokens: 0
x-ratelimit-reset-tokens: 8.64s
retry-after: 9Fix it at the source: a queue with a concurrency cap
Instead of firing every request the moment work arrives and reacting to whatever 429s come back, cap how many requests can be in flight at once and let the queue itself pace the traffic to match your actual limit.
class RateLimitedQueue {
constructor(maxConcurrent) {
this.maxConcurrent = maxConcurrent;
this.active = 0;
this.queue = [];
}
run(task) {
return new Promise((resolve, reject) => {
this.queue.push({ task, resolve, reject });
this.drain();
});
}
async drain() {
if (this.active >= this.maxConcurrent || this.queue.length === 0) return;
const { task, resolve, reject } = this.queue.shift();
this.active++;
try {
resolve(await task());
} catch (err) {
reject(err);
} finally {
this.active--;
this.drain();
}
}
}
const queue = new RateLimitedQueue(8);
const results = await Promise.all(
items.map((item) => queue.run(() => callAiApi(item))),
);
Batch instead of loop
If a pipeline loops over 500 short items and fires one API call each, that is 500 requests competing for the same requests-per-minute budget. Combining several items into a single call (several short classification tasks in one prompt, for example) can cut the request count by an order of magnitude for the same total work. For genuinely large, non-urgent volumes, OpenAI’s Batch API, introduced in 2024, offers a 50% discount on token costs in exchange for a 24-hour completion window, specifically because it lets the provider schedule your load against idle capacity instead of competing with real-time traffic.
The pipeline that worked at 50 feeds and broke at 4,000
In June 2024, a solo founder I worked with had a newsletter-summarization pipeline that processed 50 test feeds without a single error. In production, at 4,000 feeds, it broke almost immediately: roughly one in three requests came back as a 429 within the first ten minutes of the daily run, all firing in an uncapped loop the moment the job started. Adding a queue with concurrency capped to match his account’s actual RPM limit, the same pattern shown above, dropped the 429 rate to zero on the very next run. Nothing about the AI calls changed. Only the order they were allowed to happen in.
What actually fixes each limit
| Limit hit | How you will notice | Fix |
|---|---|---|
| Requests per minute | x-ratelimit-remaining-requests near zero | Queue with a concurrency cap, or space requests out |
| Tokens per minute | x-ratelimit-remaining-tokens near zero, requests quota still healthy | Batch fewer, smaller calls, or trim context size |
| Concurrent requests | 429 bursts specifically during parallel fan-out calls | Cap concurrency with a semaphore or queue |
I think teams over-invest in retry logic and under-invest in architecture. A well-tuned backoff schedule is worth having, but no amount of it fixes a pipeline that is structurally trying to move more volume through one API key per minute than the provider allows. If your 429 rate stays above a few percent after a week of running, the fix is a queue, a batch, or a tier upgrade, not one more retry attempt.
“A 429 is not the API being difficult. It is the API telling you, with the exact numbers, what your pipeline needs to change.”
Frequently asked questions
Frequently asked questions
What does a 429 error actually mean on an AI API?
It means you exceeded one of the provider’s enforced limits: requests per minute, tokens per minute, or concurrent requests. Which one specifically is not always obvious from the status code alone, check the rate-limit response headers to see which quota actually hit zero.
How do I know if I am hitting a requests-per-minute or tokens-per-minute limit?
Look at the response headers on the 429. On OpenAI, x-ratelimit-remaining-requests and x-ratelimit-remaining-tokens report each quota separately, so you can see exactly which one dropped to zero rather than guessing.
Does retrying fix a 429 error permanently?
Only if the 429 was caused by a temporary spike. If your pipeline’s sustained request or token volume exceeds your tier’s limit, retries just delay the same failure repeatedly. The permanent fix is a concurrency-capped queue, batching, or a higher usage tier.
How can I process a large batch of items without hitting rate limits?
Cap how many requests run concurrently with a queue sized to your actual RPM and TPM limits, combine several small items into fewer calls where possible, and for large non-urgent volumes, consider a provider’s batch API, which trades a completion window for a lower price and looser real-time pressure.
Can I request a higher rate limit?
Most providers scale rate limits automatically with usage tier and billing history, and some let you request a manual increase through their dashboard or support once you can show a legitimate need. Check the specific provider’s documentation, tier structures and processes differ.
Do multiple processes sharing one API key make 429s worse?
Yes. Rate limits are enforced per API key (or per organization), not per process, so several workers sharing one key all draw from the same requests-per-minute and tokens-per-minute pool. A shared concurrency cap across all of them, not just within each process individually, is what actually prevents 429s.