AI Cost Diagnostic

Work down the layers rather than starting with the easiest technique. Measure cost per completed task first so you know which workload is expensive, then look at the application layer (how many calls one finished outcome makes), then model choice, then per-request technique like caching and token discipline, and only then where inference runs. Most teams start at caching because it ships in an afternoon, and discover afterwards that the real problem was an agent loop running twelve steps when three would do.

Because they are not independent. Each lever only works on what the previous one left behind, so the savings multiply rather than add. Cutting an agent loop from twelve steps to five removes most of the tokens that caching would otherwise have discounted, so quoting both savings separately would double-count. The waterfall shows the honest sequential figure.

Because context accumulates. A twelve-step agent run does not send twelve equal requests, it sends twelve growing ones, each carrying the system prompt, the tool definitions and every prior thought and tool result. Total input tokens grow with the square of the step count, not linearly. That is why one agent run commonly costs 19x to 50x a single completion, and why the tool asks for steps per completed task rather than calls per month.

They are representative mid-2026 list prices, deliberately round, and they are editable inputs rather than fixed constants. Model pricing moves every few months, so overwrite them with the actual rates on your bill. Every figure the tool reports is derived from whatever numbers you put in those two fields.

Only above a volume where the GPU would be genuinely busy. A GPU costs the same at 9% utilisation as at 90%, and the honest calculation also includes the engineering time to run serving infrastructure, handle model updates and manage failover. Below that threshold an API is cheaper and simpler, so this tool reports no saving rather than inventing one.

What one finished user outcome costs, rather than what one API call costs. It is the number that has to sit below what the user pays you. Cost per call flatters cheap models that fail and retry: a weak model that gets it wrong three times can cost more than a single strong call. Measuring per completed task is what makes model comparisons meaningful.