RAG Cost Calculator
What your retrieval pipeline costs per query and per month, split across embedding, storage and generation, and how that compares to just using a long context window.
Usually the biggest lever in the whole system.
Questions answered, not chunks fetched.
Multiplies with top-k on every single query.
Refine the inputsHide the detail
Drives re-embedding.
Where the money goes
RAG has separate cost centres, and only one of them scales with traffic. Embedding and storage bill the same on a quiet month.
What each change is worth
Each figure is this model re-run with that one input changed. They are not additive.
- Retrieve 5 chunks instead of 20saves $614.40/mo
The single biggest lever in most RAG systems, and it usually improves answers rather than harming them: the relevant passage stops competing with near-misses.
- Halve the answer lengthsaves $70.00/mo
Output bills at four to five times input, so answer length is a pricing decision.
- Halve the chunk overlapsaves $0.103/mo
Overlap duplicates source tokens into multiple vectors, so it inflates embedding and storage together.
What actually moves a RAG bill
Three things decide it, and only the first is obvious.
Top-k times chunk size
These two numbers multiply on every query and set almost all of your generation cost. Most systems retrieve several times more than the answer needs.
Storage is a floor
Vector storage bills the same on a quiet month as a busy one. It is small for most corpora, but it is the part that does not go away when traffic does.
Reranking bills per search
Not per document. A candidate set over 100 documents costs more than one search, and chunks over ~500 tokens are split so each piece counts separately.
The full write-up is in RAG cost optimization. If your retrieval feeds an agent, the compounding cost of the loop around it is in AI agent costs.
RAG cost FAQ
How much does a RAG pipeline cost to run?+
It depends far more on top-k and chunk size than on corpus size. Generation usually dominates, because every query sends top-k chunks through the model, so retrieving twenty 512-token chunks means about 10,000 input tokens per question before the system prompt. Embedding and vector storage are typically small by comparison, but they are fixed: they bill the same whether you serve one query or a million.
What is the biggest cost driver in RAG?+
Top-k, multiplied by chunk size. Those two numbers set the input tokens on every single query, so halving either roughly halves generation cost. Most systems retrieve far more than the answer needs, and cutting from twenty chunks to five is usually the single largest saving available. It often improves answer quality too, because the relevant passage stops competing with fifteen near-misses.
Is RAG cheaper than putting everything in a long context window?+
Per query, almost always, and often by one to three orders of magnitude, because retrieval sends a few thousand tokens where long context sends the entire corpus. Published measurements put retrieval around three orders of magnitude cheaper. Prompt caching narrows the gap substantially for a stable corpus, since the stuffed context becomes a cacheable prefix billed at roughly a tenth of input price. This calculator shows both numbers.
How much does embedding a corpus cost?+
Usually far less than teams expect, and it is mostly a one-off. Embedding models are priced around a hundredth of generation models per million tokens, so a corpus of ten million tokens costs a few dollars to embed once. What recurs is churn: the share of documents that change each month and must be re-embedded. Chunk overlap raises both, because overlapping text is embedded more than once.
How is reranking priced?+
Per search rather than per document, where a search is one query plus up to about 100 documents. Two things inflate it unexpectedly. A candidate set above 100 documents counts as more than one search. And chunks longer than roughly 500 tokens are automatically split, with each piece counted as its own document, so raising chunk size can silently double the rerank bill without changing anything else.
Does a smaller chunk size reduce RAG cost?+
It reduces the tokens sent per retrieved chunk, so at a fixed top-k it lowers generation cost proportionally. But it also produces more chunks, which raises vector count and storage, and it can force a higher top-k to recover the same context, cancelling the saving. The useful target is tokens sent per query, which is top-k multiplied by chunk size, rather than either number alone.
Retrieval that answers well and bills quietly
The calculator shows the shape. We build the chunking, retrieval and caching decisions behind it. Book a free cost review.