LLM Rate Limit Planner

How to use

1. Enter your requests-per-minute (RPM) and tokens-per-minute (TPM) limits from your provider dashboard.
2. Enter the average tokens per request (input plus output, as your provider counts them) and the average time one request takes in seconds.
3. Optionally add the request rate you want to serve and a backlog size.
4. Read the maximum sustainable rate, which limit is binding, the concurrency you need, and how long the backlog takes to drain.

Worked example

A batch pipeline has a 500 RPM and 200,000 TPM limit, sends about 2,000 tokens per request and each request takes about 6 seconds. It would like 400 requests a minute and has 1,000 queued.

Result

200,000 ÷ 2,000 = 100 requests a minute by tokens, which is below the 500 RPM limit, so tokens are the bottleneck. 100 req/min = 1.67 req/s × 6 s = 10 requests in flight.

How it works

Two limits apply at once. The request limit allows RPM requests a minute; the token limit allows TPM ÷ tokens-per-request requests a minute. The sustainable rate is the smaller of the two, and the one that is smaller is the “limiting factor”.

Concurrency uses Little’s law: the average number of requests in flight equals the arrival rate multiplied by the average time each takes. At 100 requests a minute (1.67 a second) and 6 seconds each, about 10 requests are in flight. The result is rounded up.

Backlog drain time is queued requests ÷ the maximum rate per second, assuming no new arrivals and that you run exactly at the limit.

If your target rate is above the maximum, the shortfall is how many requests a minute the queue grows by. No provider data is used; every number is one you enter.

Common errors

rpm must be greater than 0.

Enter the request-per-minute limit from your dashboard.

avgTokensPerRequest must be greater than 0.

Use the average of input plus output tokens, not zero.

avgLatencySeconds must be greater than 0.

Enter the typical response time in seconds, for example 2.5.

backlog must be 0 or more.

Leave the field empty or enter 0 or a positive number.

Limits

• Averages hide variance. Bursty traffic and a few very long requests will hit limits that an average does not predict.
• Providers may measure tokens differently (some count the maximum output you request rather than what is generated) and may enforce limits per second or per day. Check your provider’s rate-limit documentation.
• Retries, concurrency caps and per-model or per-tier limits are not modelled.
• The planner assumes steady load at the limit; leave headroom in practice.

Privacy

Only numbers are processed, in your browser. Nothing is sent anywhere. No account is needed, no cookies are set and no analytics run. See thePrivacy Policy. The “Copy share link” button puts your settings after the “#” in the address; browsers do not send that part to servers, and the page’s canonical address never includes it.

FAQ

Related

• LLM Token Cost Calculator

  Tokens in, cost per call, per 1,000 calls and per month.

• Prompt Caching Calculator

  Cached vs uncached input cost, and the break-even point.

• Guide: How LLM API pricing works: input, output and cached tokens

  What you are billed for when calling an LLM API: input, output, cache write and cache read tokens, the price modifiers, and a worked monthly cost example.

Sources

• Little’s law (queueing theory), as used for the concurrency formula – checked 2026-10-14

Page last updated 2026-10-14. Written by Prajyot Mulik; see the editorial policy.