Four places your LLM API bill leaks money (and 5 free reviews)

Most LLM bills I look at waste money in the same four places. If you run a product on OpenAI, Anthropic or Gemini, check these first.

1. The same prompt prefix, sent uncached on every call

A long system prompt, tool definitions or retrieved docs that are identical on every request. Cached input is billed at roughly a tenth of the normal input price on Anthropic, and both OpenAI and Gemini discount it too.

If the fixed part of your prompt is large and you aren't caching it, this is usually the biggest single saving. On my own 7,407 coding-agent calls, caching was the difference between $6,301 and $566 at list prices. Write-up here.

2. Easy requests going to the expensive model

Classification, short rewrites, routing and "is this valid?" checks rarely need the top model. Send them to a small model and escalate only when it isn't confident.

3. Chat history re-sent in full on every turn

Cost grows with conversation length times the number of turns. Summarise or trim older turns, and cache the stable part.

4. The same request answered again and again

Repeat questions, the same URL, the same image. Hash the input and serve a saved answer.

Check your own numbers

• A rough estimate in your browser: LLM Bill Estimator
• From your logs: llm-cost-audit (open source; reads token counts only)

5 free reviews

I'm building up case studies. If you send me token counts per request for a few days (no prompt text, no user data), I'll send back where the money goes and the three biggest savings with a monthly dollar figure for each, within about 48 hours.

If you want the changes implemented afterwards, that's a separate paid job, priced as a share of what you save.

Comment below or email kri************@gmail.com with roughly what you spend a month and which models you use.

Drafted with help from Claude.