RAG Chunk Calculator
A 10,000-token document cut into 512-token chunks with a 64-token overlap makes 23 chunks and 11,408 embedding tokens, of which 12.3% are repeats. Chunks = 1 + ceil((document tokens - chunk size) ÷ (chunk size - overlap)); total = document tokens + (chunks - 1) × overlap.
How to use
- Enter the document length in tokens or characters, the chunk size and the overlap. Add your embedding price per million tokens if you want the cost.
- Read the number of chunks, the tokens you will embed in total and how many of them are repeated because of the overlap.
- To see where the cuts fall, paste the real text. It is tokenized in your browser and the first chunks are shown with the overlap highlighted.
How it works
The step between chunk starts is chunk size minus overlap. If the document fits in one chunk, the count is 1. Otherwise chunks = 1 + ceil((document tokens - chunk size) ÷ step).
Every chunk after the first repeats the last overlap tokens of the one before it, so the tokens you embed in total = document tokens + (chunks - 1) × overlap. The repeated share is (chunks - 1) × overlap ÷ that total. The last chunk is whatever is left, so it is usually shorter.
Characters are converted at 4 characters per token, the rule of thumb OpenAI gives for English text. Other languages, code and numbers use more tokens per character, so paste the text for an exact count.
Pasted text is tokenized in your browser with the real byte-pair-encoding tables (js-tiktoken). The preview cuts the token list into the same windows as the formula and decodes each one. Fixed-size token windows ignore sentence and paragraph boundaries; splitters that respect them make uneven chunks and a different count.
Cost = embedding tokens ÷ 1,000,000 × the price per million tokens you enter. Take the price from your embedding provider's pricing page.
FAQ
What chunk size and overlap should I use?
There is no universal value. Sizes from a few hundred to about a thousand tokens are common, with an overlap of roughly 10 to 20 percent. Smaller chunks give more precise matches but more vectors to store; larger chunks keep more context but can blur the match. Test with your own questions.
Does overlap raise the embedding bill?
Yes, each overlap is embedded twice. For a 10,000-token document in 512-token chunks with 64 tokens of overlap, 1,408 of the 11,408 tokens sent (12.3%) are repeats, so you embed 14.1% more than the document itself.
Are the chunk counts exact?
They are exact for the rule shown: fixed windows of tokens. Libraries that split on sentences, paragraphs or headings make chunks of uneven size, so their counts differ. The conversion from characters to tokens is an estimate.
Which tokenizer should I pick?
The one your embedding model uses. OpenAI's text-embedding-3 models use cl100k_base. Models from other providers have tokenizers that are not available here, so the count is a close reference.
Is my text sent anywhere?
No. Counting and the preview run in your browser. The tokenizer table is downloaded from this site once, and your text is never uploaded.