The Real Cost of Running Your Own LLM vs Calling an API

"Just self-host an open model, it's free." I've heard this a lot, and it's one of the most expensive sentences in tech. Open weights are free. Running them is not. Here's the honest cost comparison between hosting your own LLM and just calling an API, with real 2026 numbers.

The Short Answer

For most teams, calling an API is cheaper, and it stays cheaper until you're pushing serious volume - roughly 5 to 10 million tokens a month on a premium model. Past that, self-hosting can win on raw compute, but hidden costs like engineering time often erase the savings anyway. Scale and specific needs decide it, not vibes.

How API Pricing Works

The API model is simple: you pay per token, and someone else owns the hardware. Prices in 2026 keep falling:

• Claude Sonnet 5 runs about $2 per million input tokens and $10 per million output.
• OpenAI's GPT-5.6 family starts near $1 per million input.
• Meta's Muse Spark (a cheaper frontier model) sits around $1.25 input and $4.25 output.

The appeal is that you pay for exactly what you use and nothing when you're idle. No servers to babysit, no 3 a.m. GPU failures. For spiky or low-volume workloads, that's hard to beat.

How Self-Hosting Actually Costs Money

Self-hosting flips the model. You rent or buy GPUs and pay for them whether you're using them or not. A single NVIDIA H100 runs about:

• $2 to $3 per hour on specialized GPU clouds like Lambda, RunPod, or CoreWeave
• Closer to $7 per hour on AWS
• $12 per hour on Azure

Run one H100 around the clock and you're looking at roughly $1,800 to $2,000 a month - for one GPU. Serious models need more than one. That's the number people quote. It's also the number that lies to you.

The Break-Even Math

Let's do the honest comparison. On a single on-demand H100 running a 70-billion-parameter open model efficiently, self-hosting starts to beat premium API pricing somewhere around 5 to 10 million tokens per month, or very roughly 500,000 tokens a day sustained.

• Below that, the API wins easily, because you'd be paying $2,000 a month for a GPU that mostly sits idle.
• Above it, and especially at 100 million-plus tokens a month, the math tilts hard toward self-hosting, where large deployments can save real money.

But that's only the compute line.

The Costs Nobody Puts in the Spreadsheet

Here's where the "free model" story falls apart. Raw GPU cost is only about 30 to 40 percent of the true cost of running your own LLM. Plan for a 2.5 to 3x multiplier once you add networking, storage, redundancy, and monitoring.

Then there's the big one: people. Keeping a self-hosted model reliable takes real engineering. Estimates put a minimum viable team at 1.5 to 2 full-time engineers, which is $270,000 to $550,000 a year in salary alone. That "free" open-source model can quietly cost half a million dollars a year in engineering time before you count a single GPU.

An API has none of that. The provider eats the ops burden. You're paying a premium per token precisely so you don't need that team.

When Self-Hosting Actually Wins

It's not always about cost. Self-hosting makes sense when:

• You're at real scale - processing tens or hundreds of millions of tokens a month consistently, so the GPUs stay busy. Idle GPUs are what kill the economics.
• You have hard data or compliance requirements - where sending data to a third-party API isn't allowed. Sometimes control, not cost, is the reason.
• You need predictable latency or heavy customization that a shared API can't give you.

If none of those apply, you're probably paying a lot to feel in control.

The Hybrid Reality

Most teams that get this right don't pick one and commit forever. They start on APIs because it's fast and cheap at low volume, then move the highest-volume, most stable workloads to self-hosting once the numbers clearly justify it, while keeping everything else on APIs. Cost follows usage instead of ideology.

The Bottom Line

Open weights are free. Running an LLM is not. APIs win for most teams because you skip the GPUs, the ops, and the 2 extra engineers, and you only pay for what you use. Self-hosting wins at real scale or when compliance forces your hand, but only if you count the full cost - not just the GPU sticker price. Do the token math first, add the hidden 60 percent, and you'll make the call for the right reasons instead of the "it's free" myth.