Token Economics: Why Your LLM Bill Is 3 What the Pricing Page Promised

The Five Leaks, at a Glance

These aren't additive. They're stackable. Combined, the difference between naive pricing and optimized reality is 40–65%.

Leak 1: Workload Ratio - Your Use Case Is the Multiplier

Output tokens cost 3–5× more than input tokens. The ratio between them is determined by your workload - and it's the single largest cost variable.

Same model (GPT-4o). Same request count (10,000/day). Different workloads:

2.9× spread - same model, same request count. Before comparing providers. Before factoring any other leak.

Fix: Measure your actual input-to-output token ratio in production. Most teams guess 1:1. Almost no real workload is 1:1.

Leak 2: Tokenizer Variance - You're Comparing Different Units

Every provider's tokenizer is different. The same text produces different token counts on each:

Why this matters: comparing per-token prices without benchmarking your actual text = comparing different units. Provider A at $2.00/M with a 10% hungrier tokenizer = Provider B at $2.20/M. The cheaper sticker price may be more expensive after tokenization.

Fix: Run your actual production text through 2–3 candidate tokenizers before committing. At 1M+ requests/day, a 10% efficiency gap is thousands/month.

Leak 3: Prompt Caching - The 90% Discount Nobody Turns On

Anthropic introduced prompt caching in August 2024. OpenAI followed with automatic caching. Google launched context caching in early 2025. The discounts are the largest cost lever in LLM APIs - and most teams never configure it.

What's actually cacheable in your app:

Real example: a customer support chatbot with 1,500-token system prompt, 1,000-token few-shot examples, 3,000-token RAG context per query. Total input: 5,500 tokens. Cacheable: 2,500 tokens (45%).

Fix: Identify your cacheable prefix tokens. Structure API calls so they appear at the beginning of every prompt. Anthropic requires explicit cache point marking; OpenAI and Google handle it automatically.

Leak 4: Batch Processing - Half Price, No Catch

OpenAI and Anthropic offer batch endpoints at 50% off standard pricing. The tradeoff: up to 24-hour completion SLA instead of real-time response. For offline workloads - evaluation runs, dataset labeling, embedding generation, nightly summarization, synthetic data generation - there is literally zero downside. The 50% discount is free money.

Stacked with prompt caching:

• Cached input + batch = 5% of sticker price (90% off × 50% off)
• Uncached input + batch = 50% of sticker price
• Output + batch = 50% of sticker price

Moving 60% of the support chatbot's traffic to batch: $55,572/year vs $104,025 naive = 47% saved.

Fix: Segment traffic into realtime and async. Route async to batch endpoints. The infrastructure change is an API endpoint swap - no model changes, no prompt changes.

Leak 5: Rate Limit Retries - Paying Twice

When your app hits API rate limits, the client retries - and the failed tokens are charged. At 2% retry rate, 10,000 requests/day: $365/year in wasted input tokens. Small, but the architectural cost is larger: teams over-provision multiple providers to avoid limits.

Fix: Exponential backoff with jitter. Monitor retry rate (if >1%, you need higher limits or a queuing layer). Route async traffic to batch endpoints (separate, higher limits).

The 2026 Provider Landscape

The caching twist: Anthropic's 90% cache discount makes Claude Opus 4's effective cached input ($1.50/M) cheaper than GPT-4o's standard input ($2.50/M). At high cache hit rates, the premium tier beats the standard tier on price.

Self-Hosted vs API: The Breakeven Math

The utilization reality: most teams overestimate their GPU utilization. Self-hosted GPUs idle during nights, weekends, holidays. The API charges zero for idle time. Bursty traffic → API wins. Steady high throughput → self-hosting wins.

The hidden cost: self-hosting a 70B model across 3 GPUs requires understanding tensor parallelism, quantization (AWQ/GPTQ/FP8), continuous batching (vLLM/TGI), and GPU node management. Budget 0.25–0.5 FTE for production self-hosting.

Five Questions That Determine Your Bill

1. What's your actual input-to-output ratio? Measure it. Don't guess 1:1.
2. What % of input tokens are cacheable? If >20%, Anthropic's 90% cache discount may beat GPT-4o despite the higher sticker.
3. What % of traffic tolerates 24-hour latency? Batch = 50% off. Moving 30% of traffic to batch cuts blended cost by 15%.
4. Is traffic steady or bursty? Steady → self-host. Bursty → API. Be honest about utilization.
5. Need multi-provider for reliability? A three-tier routing strategy (budget/standard/flagship) cuts blended per-token cost by 60–80% vs routing everything to the flagship.

Interactive calculator: jslet.com/llm-api-pricing-calculator - compare 12 models across 6 providers with caching, batch, and workload presets. All client-side, no signup.