AI Cost Diagnostic
Estimate what your workload costs today, then see which of the five cost levers saves you the most. Applied in order, because each one only works on what the last left behind.
The model loops with tools. Every step re-sends the conversation.
Model calls it takes to finish one job.
Finished outcomes, not raw API calls.
Refine the inputsHide the detail
System prompt, tools, fixed docs.
Genuinely new input each step.
Output usually costs 4x to 5x input.
Representative mid-2026 list prices. Overwrite them with your actual rates.
A best case. It assumes every lever lands cleanly and quality holds. Treat the first one or two as plannable and the rest as direction.
Biggest first move on your numbers: $4,877 a month.
The levers, in order, on your numbers
Each row acts on what the row above it left behind, which is why the order matters and why the savings do not add up to the total.
- 1Measure cost per completed taskno saving
No direct saving. Without it the levers below are guesses about which workload is actually expensive.
Cost per completed task - 2Fix the application layersaves $4,877/mo
Assumes iteration caps and circuit breakers remove ~20% of steps as retry overhead, and that carrying a summary plus the last two turns replaces the full transcript (~65% less carried context).
AI agent costsBill after this lever: $7,243/mo
- 3Right-size the modelsaves $4,106/mo
Assumes 70% of steps in this workload are narrow enough for a small model, with 15% of those escalating back to the frontier tier on low confidence or a schema failure.
Small models vs LLMsBill after this lever: $3,138/mo
- 4Make each request cheapersaves $1,548/mo
Assumes 90% of the stable prefix becomes cache reads at a tenth of input price, capping max_tokens and asking for structured output trims 20% of output tokens.
Prompt cachingBill after this lever: $1,589/mo
- 5Decide where inference runssaves $556.18/mo
Assumes a self-hosted open-weight model at high GPU utilisation, net of DevOps time. Deliberately conservative because it overlaps with routing: traffic already moved to a small model has less left for self-hosting to save. The most uncertain number here.
Self-hosting vs APIBill after this lever: $1,033/mo
What this model assumes
Three things drive the numbers above. Each lever states its own assumption inline, so you can disagree with any of them and see the effect.
Context compounds
Every agent step re-sends the conversation so far, so input tokens grow with the square of the step count. This is usually the largest line on the bill and the one nobody instruments.
Order beats technique
Levers multiply, they do not add. Caching a request you should never have sent saves 90% of a number that should be zero, which is why the tool reports a waterfall.
Per task, not per call
A cheap model that fails and retries three times costs more than one strong call. Cost per completed task is the only figure that survives a model swap.
This is the diagnostic front end to the full playbook on LLM cost optimization. For the caching lever specifically, size it with the prompt caching calculator.
AI cost diagnostic FAQ
How do I know which AI cost lever to pull first?+
Work down the layers rather than starting with the easiest technique. Measure cost per completed task first so you know which workload is expensive, then look at the application layer (how many calls one finished outcome makes), then model choice, then per-request technique like caching and token discipline, and only then where inference runs. Most teams start at caching because it ships in an afternoon, and discover afterwards that the real problem was an agent loop running twelve steps when three would do.
Why does this tool apply the levers in order instead of adding up percentages?+
Because they are not independent. Each lever only works on what the previous one left behind, so the savings multiply rather than add. Cutting an agent loop from twelve steps to five removes most of the tokens that caching would otherwise have discounted, so quoting both savings separately would double-count. The waterfall shows the honest sequential figure.
Why are agents so much more expensive than a single call?+
Because context accumulates. A twelve-step agent run does not send twelve equal requests, it sends twelve growing ones, each carrying the system prompt, the tool definitions and every prior thought and tool result. Total input tokens grow with the square of the step count, not linearly. That is why one agent run commonly costs 19x to 50x a single completion, and why the tool asks for steps per completed task rather than calls per month.
Are the prices in this tool current?+
They are representative mid-2026 list prices, deliberately round, and they are editable inputs rather than fixed constants. Model pricing moves every few months, so overwrite them with the actual rates on your bill. Every figure the tool reports is derived from whatever numbers you put in those two fields.
When is self-hosting an LLM actually cheaper than an API?+
Only above a volume where the GPU would be genuinely busy. A GPU costs the same at 9% utilisation as at 90%, and the honest calculation also includes the engineering time to run serving infrastructure, handle model updates and manage failover. Below that threshold an API is cheaper and simpler, so this tool reports no saving rather than inventing one.
What is cost per completed task?+
What one finished user outcome costs, rather than what one API call costs. It is the number that has to sit below what the user pays you. Cost per call flatters cheap models that fail and retry: a weak model that gets it wrong three times can cost more than a single strong call. Measuring per completed task is what makes model comparisons meaningful.
Cut your AI bill without cutting quality
The diagnostic shows which lever to pull. We build the instrumentation, loop controls and routing that capture it. Book a free cost review.