Self-Host vs API Break-even

On representative mid-2026 numbers, an 8B open-weight model on a specialist-cloud H100 breaks even against a mid-tier API somewhere around 800 million output tokens a month, once you include two replicas for redundancy, 30% average utilisation and twenty hours of engineering time. Below that the API wins comfortably. The exact figure moves a lot with your GPU provider and utilisation, which is what this calculator is for.

Because a GPU bills the same whether it is at 9% load or 90%. Traffic is peaky, so the capacity you can actually use is far below the capacity you buy, and paying for idle silicon is the single most common reason a self-hosting business case fails in practice. At 30% utilisation you are paying roughly three times the notional cost per token you calculated from the spec sheet.

Enormously, and usually more than which GPU you pick. The same H100 runs around $3 an hour on specialist GPU clouds and $11 to $13 an hour on the major hyperscalers, roughly a 4x spread for identical silicon. That difference alone can move the break-even volume by a factor of three, so the provider decision often settles the build-versus-buy question before any other variable does.

Yes, and leaving it out is the most common way these business cases get inflated. Serving infrastructure needs deployment, model updates, capacity planning, failover, monitoring and someone on call. At small scale those hours frequently exceed the GPU bill itself, which is why self-hosting rarely pays for a team serving modest volume no matter how cheap the hardware looks.

Usually not, unless volume is very large or you have a requirement that rules out APIs entirely. A 70B model needs several GPUs per replica and serves far fewer tokens per second than an 8B one, so cost per token rises on both axes at once. Against a cheap mid-tier API it often never breaks even. Against frontier pricing the case is much easier.

Data residency, predictable spend, no rate limits, no vendor deprecation cycle, and the ability to fine-tune freely on your own data. Those can justify self-hosting well below the cost break-even, and if any of them are hard requirements the arithmetic here is not really the deciding factor. What the calculator prevents is choosing it for cost reasons that do not hold.