Tokens per Second Calculator
Response time = TTFT + output tokens ÷ tokens per second, so 1,000 tokens at 50 tok/s with 0.5 s TTFT take 20.5 s. A local model cannot decode faster than memory bandwidth ÷ model size: 307 GB/s (Apple M5 Pro) ÷ 4.9 GB ≈ 63 tok/s.
How to use
- Choose Response time to get seconds from a speed, or Required speed to get the tokens per second for a target time.
- Enter the output tokens and the time to first token (TTFT). TTFT covers network, queueing and reading the prompt.
- For a local model, pick a device or type its memory bandwidth, and enter the model file size in GB. The second result is the theoretical ceiling for generation speed.
How it works
Response time = TTFT + output tokens ÷ tokens per second. 1,000 tokens at 50 tokens per second with 0.5 s TTFT take 0.5 + 20 = 20.5 seconds.
Required speed = output tokens ÷ (target time − TTFT). To finish 1,000 tokens in 10.5 seconds with 0.5 s TTFT you need 100 tokens per second. If the target is not longer than TTFT, no speed is enough.
Local upper bound = memory bandwidth (GB/s) ÷ model size (GB). With batch size 1, each new token reads every weight from memory once, so decoding is limited by bandwidth rather than compute. 307 GB/s and a 4.9 GB model give at most about 62.7 tokens per second.
Limits of the bound: real speed is lower because kernels never reach full bandwidth and the KV cache is read too, more so at long contexts. Mixture of experts models read only their active weights per token, so use the active size there. Prompt processing is compute bound and not covered. Bandwidth figures: nvidia.com H100 page (SXM 3.35 TB/s, NVL 3.9 TB/s) and apple.com MacBook Pro specs (M5 153, M5 Pro 307, M5 Max 460 or 614 GB/s), checked 16 October 2026.
Time for 1,000 output tokens
| Tokens per second | Generation time | With 0.5 s TTFT |
|---|---|---|
| 20 | 50 s | 50.5 s |
| 50 | 20 s | 20.5 s |
| 100 | 10 s | 10.5 s |
| 200 | 5 s | 5.5 s |
Bandwidth ceiling for a 4.9 GB model
A model of about 4.9 GB is an 8B model in Q4_K_M (see the quantization size calculator). On Apple M5 the ceiling is about 31 tokens per second, on M5 Pro about 63, on M5 Max with the 40-core GPU about 125, and on an H100 SXM about 684. Measured speeds land below these figures.
FAQ
How long does a 1,000 token answer take?
At 50 tokens per second it takes 20 seconds of generation, plus the time to first token. A 300 to 500 token answer takes 6 to 10 seconds at that speed.
How fast will a model run on my Mac?
Divide the memory bandwidth by the model size. An M5 Pro at 307 GB/s with a 4.9 GB Q4 model has a ceiling of about 63 tokens per second, and real speed is lower.
Why do smaller quantizations generate faster?
Fewer bytes are read for each token. Halving the model size roughly doubles the bandwidth ceiling, which is why Q4 runs faster than Q8 on the same machine.
Does this cover prompt processing?
No. Reading the prompt is compute bound and is part of TTFT. Enter a measured TTFT for long prompts.