RAG Cost Calculator

Because most of them price only embedding and vector storage. Generation is usually most of what a RAG pipeline costs to run each month, often over 90% at production query volumes, and a calculator that leaves it out will understate your bill many times over. This one prices all three cost centres, plus reranking, and shows the split so you can see which one to attack.

It depends far more on top-k and chunk size than on corpus size. Generation usually dominates, because every query sends top-k chunks through the model, so retrieving twenty 512-token chunks means about 10,000 input tokens per question before the system prompt. Embedding and vector storage are typically small by comparison, but they are fixed: they bill the same whether you serve one query or a million.

Top-k, multiplied by chunk size. Those two numbers set the input tokens on every single query, so halving either roughly halves generation cost. Most systems retrieve far more than the answer needs, and cutting from twenty chunks to five is usually the single largest saving available. It often improves answer quality too, because the relevant passage stops competing with fifteen near-misses.

Per query, almost always, and often by one to three orders of magnitude, because retrieval sends a few thousand tokens where long context sends the entire corpus. Published measurements put retrieval around three orders of magnitude cheaper. Prompt caching narrows the gap substantially for a stable corpus, since the stuffed context becomes a cacheable prefix billed at roughly a tenth of input price. This calculator shows both numbers.

Usually far less than teams expect, and it is mostly a one-off. Embedding models cost a small fraction of generation models per million tokens, so a corpus of ten million tokens costs about $0.23 to embed once with text-embedding-3-small, or about $1.50 with text-embedding-3-large. What recurs is churn: the share of documents that change each month and must be re-embedded. Chunk overlap raises both, because overlapping text is embedded more than once.

Per search rather than per document, where a search is one query plus up to about 100 documents. Two things inflate it unexpectedly. A candidate set above 100 documents counts as more than one search. And chunks longer than roughly 500 tokens are automatically split, with each piece counted as its own document, so raising chunk size can silently double the rerank bill without changing anything else.

It reduces the tokens sent per retrieved chunk, so at a fixed top-k it lowers generation cost proportionally. But it also produces more chunks, which raises vector count and storage, and it can force a higher top-k to recover the same context, cancelling the saving. The useful target is tokens sent per query, which is top-k multiplied by chunk size, rather than either number alone.