TL;DR
Cost per task is total spend divided by the number of tasks that actually finished. It is the only AI cost metric that survives a model swap, because it prices failure.
Token pricing does not. A model with cheaper tokens that fails more often can cost more per finished job than an expensive model that gets it right first time. The Register reported exactly this in July 2026: one model landing in the top capability tier at $1.28 per task against a rival's $1.94, despite the token comparison pointing the other way.
The advice to "measure cost per task instead of cost per token" is now everywhere. What is missing is how. This post is the how: the formula, the five fields you have to log, what the word completed has to mean before the number means anything, and why you read the 95th percentile rather than the average.
Want this instrumented rather than estimated? We put cost per completed task in before the model choice on every build. See how we approach AI development.
Key Takeaways
- Cost per task = total spend divided by tasks that met your success bar. Failed attempts stay in the numerator.
- A cheaper model that fails 20% of the time can cost more per completed task than one that fails 2%.
- The definition of "completed" does most of the work. Without a success bar, the metric is just spend per attempt.
- Read the 95th percentile, not the mean. AI cost is long-tailed and the mean hides the runs that hurt.
- The same task costs wildly different amounts depending on architecture: one call, retrieval, or an agent loop.
- The number is only good or bad relative to what a user pays you for that task.
What is cost per task?
Cost per task, also written cost per completed task, cost per successful task or cost per completion, is:
total spend over a period, divided by the number of tasks that finished successfully in that period
Two things make it different from every other cost metric, and both come from the denominator.
Failed work stays in the numerator. If three attempts were needed to produce one good answer, all three are billed and the denominator counts one. That single property is what makes the metric honest, because it is the only way retries and loops show up in a cost number at all.
It is denominated in something a business recognises. Tokens are an implementation detail. A completed task is a support ticket resolved, a document extracted, a PR reviewed. That is the unit you can put next to a price.
OpenAI's own framing in its July 2026 scorecard puts it as depending on "price, the amount of compute used, and the likelihood of reaching the right result". That third term is the one token pricing throws away.
Why token price lies
Here is the case that makes the metric necessary rather than merely nicer.
Take a task where a cheap model succeeds 70% of the time and a strong model succeeds 97%. The cheap model is 15x cheaper per call. Intuition says use the cheap one. Now add the retry behaviour that every production system has: on failure, try again.
The rule worth memorising: the break-even success rate is roughly one divided by how many times cheaper the model is. A model 15x cheaper only needs to succeed about 1 in 15 times as often to stay ahead, which is a low bar. A model 2x cheaper needs to be within half, which is not.
That is why the answer is usually "route" rather than "replace", and it is why a cheap model inside an agent loop is far more dangerous than a cheap model on a single call: in a loop, a failure does not just produce a worse answer, it triggers a retry that re-sends the whole accumulated context. The compounding is worked through in AI agent costs.
How to actually compute it
This is the part the rest of the internet skips. It is not complicated, but it has to be done at the right layer.
Log five fields on every model call, at the point of invocation rather than in a wrapper that only sees successes:
| Field | Why |
|---|---|
input_tokens, output_tokens, cached_tokens |
The three bill separately and cached is roughly a tenth |
model |
So a routing change is visible in the data |
task_id |
The join key. Without it you have call costs, not task costs |
attempt_number |
Retries are invisible without it, and they are the point |
feature |
So you can attribute a rise to something you shipped |
Then aggregate by task_id, not by call. Sum the cost of every attempt that carries the same task id, and divide by the count of task ids that reached a successful terminal state.
The task_id is where most implementations fail. A cost dashboard that groups by API call gives you spend per attempt, which is the metric that improves when quality degrades. The join key is what makes the number honest.
Include the costs that are not tokens. Retrieval, reranking, embedding, and any human review time attached to failures all belong in the numerator if you want a number you can compare against revenue.
The word "completed" is doing most of the work
A cost per task number is only as meaningful as the success bar behind it, and this is where teams quietly cheat without noticing.
If "completed" means "the API returned 200", you are measuring spend per attempt with extra steps. The bar has to be something a user would recognise as done: the extraction validated against the schema, the answer cited a real source, the ticket was not reopened, the PR was merged.
Three practical ways to define it, in ascending order of effort and value:
- Structural: the output parsed and passed validation. Cheap, catches the worst failures, misses confidently wrong answers.
- Behavioural: the user did not retry, rephrase, escalate or abandon. Free once you have the events, and a strong signal.
- Evaluated: an evaluation set or a judge model scored it. Most work, most trustworthy, and the only option for open-ended generation.
Pick one and write it down. A cost per task figure without a stated success bar is not comparable to anyone else's, including your own from last quarter.
Read the distribution, not the average
The mean is the wrong statistic for this metric and it will mislead you consistently.
AI cost per task is long-tailed by nature. Most runs are cheap and finish quickly. A minority hit a retry loop, carry an oversized context, or grind through many more steps than intended, and those are the ones on the invoice.
Report both. The median tells you what a normal run costs, which is what you use for pricing. The 95th percentile tells you what your worst runs cost, which is what you use for engineering. A rising p95 with a flat median is the signature of a loop problem, and it is invisible in an average.
The same task, three architectures
Cost per task is not a property of a model. It is a property of the system around it, and the spread across architectures is far larger than the spread across model tiers.
| Architecture | What drives the cost | Where the number comes from |
|---|---|---|
| Single call | Prompt size and answer length | Almost entirely token discipline |
| Retrieval | top-k multiplied by chunk size, on every query | RAG cost optimization |
| Agent loop | Steps, and context re-sent on each one | AI agent costs, often 19x to 50x a single call |
| Self-hosted | GPU hours and utilisation, not tokens | Self-hosting vs API |
That last row is worth pausing on, because it breaks the intuition the whole metric is built on. On self-hosted infrastructure your marginal cost per task approaches zero while your fixed cost does not move at all. Cost per task on owned hardware is really total infrastructure divided by throughput, which means the way to improve it is to run more tasks rather than to make each one cheaper.
When the number is bad: AI unit economics
A cost per task figure is not good or bad on its own. It is good or bad relative to what someone pays you for that task.
This is where the metric earns its place in a business conversation rather than an engineering one. AI unit economics is simply this comparison done honestly: cost per completed task against revenue per task, at the volume a real customer generates.
Three outcomes, and only one of them is an engineering problem:
- Cost per task well below revenue per task. Healthy. Grow.
- Cost per task close to revenue per task. An engineering problem. The levers, in order, are in our guide to LLM cost optimization.
- Cost per task above revenue per task at any plausible efficiency. A pricing or product problem. No amount of caching fixes a feature that loses money on every use, and teams routinely spend months optimising when the answer was to change the plan or cap the usage.
The related number for a subscription business is AI cost per customer: cost per task multiplied by tasks per customer per month. That is the figure that has to sit under your gross margin, and it is the one that moves when engaged users get more engaged.
How we instrument this at HorizonLux
HorizonLux is a software development company specialising in AI, and this metric is the first thing we put into a build rather than the last.
- The cost record goes in before the model choice. Five fields at the point of invocation, including
attempt_number, because a design decision made without this number is a guess. - The success bar is written down in the first week. Structural at minimum, behavioural where the events exist, evaluated where the output is open-ended.
- We report median and p95, never the mean. A rising p95 against a flat median is a loop problem and we want to see it in week one, not on the invoice.
- Cost per task is a release gate. If a change moves it the wrong way, that is a regression, treated like any other.
- We compare against revenue per task, because the point of the number is to sit next to a price.
What to do this week
- Add
task_idandattempt_numberto your model call logs. Without those two fields none of the rest is possible. - Write down what "completed" means for your main workload. One sentence. Put it in the repo.
- Compute median and p95 cost per completed task for last month. Do not compute the mean.
- Put it next to revenue per task. If the gap is uncomfortable, you now know whether it is an engineering problem or a pricing one.
- Make it a release gate, so the number cannot quietly drift.
Cost per token is a price. Cost per task is a fact about your system. The first is set by someone else and the second is entirely yours, which is why it is the only one worth managing. If you are still working out which part of your stack is responsible for the number, start with why is my AI bill so high.
Want this in place before the first invoice? We instrument cost per completed task from the first commit and gate releases on it. See how we approach AI development, or tell us what your invoice looks like.
Frequently asked questions
What is cost per task in AI?
Cost per task is total spend divided by the number of tasks that finished successfully. Failed attempts stay in the numerator, which is what makes it different from cost per call: three attempts producing one good answer counts as three costs and one task. It is denominated in something a business recognises, such as a resolved ticket or an extracted document, so it can sit directly next to a price.
How do I calculate cost per completed task?
Log input, output and cached tokens, the model, a task_id, an attempt_number and the feature on every model call at the point of invocation. Then sum the cost of all attempts sharing a task id, and divide by the count of task ids that reached a successful terminal state. The task_id is the field most implementations miss, and without it you are measuring spend per attempt rather than per task.
Why is cost per token a misleading metric?
Because it ignores whether the call worked. A model with cheaper tokens that fails more often can cost more per finished job than an expensive one that succeeds first time, since every failed attempt is billed in full. The Register documented a case in July 2026 where a model reached the top capability tier at $1.28 per task against a rival's $1.94, a ranking token pricing alone would have reversed.
Should I use the average or the percentile?
The 95th percentile, alongside the median. AI cost per task is long-tailed: most runs are cheap and a small fraction cost many times the median because of retry loops or oversized context, and those runs are what fill the invoice. A rising p95 against a flat median is the signature of a loop problem, and an average conceals it completely.
What counts as a completed task?
Whatever a user would recognise as done, defined in advance and written down. Structural definitions check the output parsed and validated. Behavioural definitions check the user did not retry, rephrase or escalate. Evaluated definitions score against an evaluation set or judge model, which is the only workable option for open-ended generation. Without a stated bar the number is not comparable to anything, including your own figure from last quarter.
What is AI unit economics?
Cost per completed task compared honestly against revenue per task at realistic volume. Well below revenue is healthy. Close to it is an engineering problem with known levers. Above it at any plausible efficiency is a pricing problem that optimisation cannot fix. For subscriptions the related figure is AI cost per customer: cost per task multiplied by tasks per customer per month.
Does cost per task work for self-hosted models?
It works but it inverts. On owned GPUs the marginal cost of one more task approaches zero while the fixed infrastructure cost does not change at all, so cost per task becomes total infrastructure divided by throughput. That means the way to improve it is to push more work through the same hardware rather than to make each task cheaper, which is the opposite of the advice that applies on a metered API.
