Self-Host vs API Break-even

Later than most teams expect. On September 2026 prices, an 8B open-weight model on a $3.49/hour H100 breaks even against the budget API it would replace (GPT-5.6 Luna, $0.20 in / $1.20 out) at around 36 billion output tokens a month, once you include two replicas, 30% average utilisation and twenty hours of engineering time. On AWS on-demand GPUs it never does. Compare it with a pricier API and it wins far sooner, but an 8B model does not replace a mid-tier one. The exact figure moves with your GPU price and utilisation, which is what this calculator is for.

Because a GPU bills the same whether it is at 9% load or 90%. Traffic is peaky, so the capacity you can actually use is far below the capacity you buy, and paying for idle silicon is the single most common reason a self-hosting business case fails in practice. At 30% utilisation you are paying roughly three times the notional cost per token you calculated from the spec sheet.

A lot, and often more than which GPU you pick. The same H100 cost $3.49 an hour on RunPod and $6.88 on AWS on-demand in September 2026, about a 2x spread for identical silicon. In the default scenario that difference turns a break-even of 36 billion tokens a month into no break-even at all, so the provider decision can settle the build-versus-buy question before any other variable does.

Yes, and leaving it out is the most common way these business cases get inflated. Serving infrastructure needs deployment, model updates, capacity planning, failover, monitoring and someone on call. At small scale those hours frequently exceed the GPU bill itself, which is why self-hosting rarely pays for a team serving modest volume no matter how cheap the hardware looks.

Usually not, unless volume is very large or you have a requirement that rules out APIs entirely. A 70B model needs several GPUs per replica and serves far fewer tokens per second than an 8B one, so cost per token rises on both axes at once. Against a budget API such as Gemini 3.8 Flash it never breaks even in the default scenario. Against frontier pricing the case is much easier, but check the open model actually matches the frontier one on your task.

Data residency, predictable spend, no rate limits, no vendor deprecation cycle, and the ability to fine-tune freely on your own data. Those can justify self-hosting well below the cost break-even, and if any of them are hard requirements the arithmetic here is not really the deciding factor. What the calculator prevents is choosing it for cost reasons that do not hold.