Our AI coding bill quietly tripled. Here's what we learned fixing it.

A few months ago I opened our cloud bill and had that small stomach drop moment every engineer knows. Our AI coding spend had roughly tripled. Not because anything was broken, but because everything was working. The team had gone all in on Claude Code and Codex, they were shipping faster than ever, and nobody, including me, could say where the money was actually going.

I run engineering at a small software company. We're not Uber. But it turns out the shape of this problem is the same whether you have 15 engineers or 5,000, and the big players hit it first and hardest. Uber reportedly burned through its entire annual AI coding budget in four months. Meta's engineers pushed tens of trillions of tokens in a single month, partly chasing an internal usage leaderboard, and their own CTO had to point out that token usage isn't a measure of impact. If it can happen to them, it can absolutely happen to you.

Here's what actually went wrong for us, and the concrete things that brought it back under control.

The problem isn't the tools. It's the blindness.

The productivity gains from agentic coding are real. I'm not here to be a skeptic. The speedup was immediate and nobody wanted to go back. The problem is a specific and dangerous gap: you can't see the spend until the invoice arrives, and by then it's already spent.

When I finally put real visibility on our usage, the waste wasn't dramatic. There was no villain. It was ordinary, invisible, and constant:

• One developer was quietly burning more tokens than five others combined. Not maliciously, just a workflow that leaned hard on the most expensive model for everything.
• Routine tasks like doc generation, cleanup, and commit messages were hitting a frontier model when a model a fraction of the price would have produced identical output.
• A test and fix loop someone kicked off had been running semi attended for the better part of two weeks.

Individually, each of these is nothing. Together, they were the entire gap between "the savings AI promised" and the bill I was actually staring at.

The root cause came down to three blind spots:

• No usage visibility. The bill is one big number. You can't break it down by person, project, or task, so you can't reason about it.
• No control. Nothing stops a runaway process or a bad default until the money is gone.
• No model governance. The tools pick the model themselves. You pay for that choice with no say in it.

What actually reduced the cost

Once I could see the problem, fixing it was mostly mechanical. Here's what moved the needle, roughly in order of impact.

1. Prompt caching (the biggest single lever)

This is the one most teams leave on the table, and it's the largest. Agentic coding re sends a huge amount of stable context every turn: the system prompt, tool definitions, the same files. With prompt caching, that repeated prefix reads at a fraction of full input price, and cached tokens don't count against your rate limits either.

The critical gotcha: if you route Claude Code through any kind of proxy, it's dangerously easy to break caching by not forwarding the right headers or by mutating the request body. A misconfigured gateway can silently turn every request back into full price. If you cache correctly, this alone can cut input costs dramatically.

Check: confirm your cache read token counts are non zero on repeat requests. If they're zero, caching isn't actually happening.

2. Right size the model per task

Not everything needs the frontier model. Doc generation, summaries, simple refactors: a cheaper model handles them at identical quality for a fraction of the cost. The spread is enormous. The same prompt across top models can range from cents to several dollars.

Codex and Claude Code both have a "small fast model" concept for background tasks, so make sure that's actually pointed at a cheap model, not silently inheriting your expensive default.

3. Cap the loops

The single scariest line item is anything that calls a model in a loop unattended. Agent runs, test fix cycles, background jobs. Put a hard ceiling on them, a spend or token cap that stops the process rather than a memo asking people to be careful. Make it a config, not a policy document.

4. Trim what you send

Beyond caching, a lot of what gets sent to the model is stale context being dragged along turn after turn. Compressing bulky, old tool output before it hits the model can cut token volume substantially on long agent sessions, as long as you leave recent context and reasoning untouched.

Be careful here. Aggressive trimming of active context breaks agentic workflows, so this is a "squeeze the old stuff only" move.

5. Make spend visible to the people spending it

This one is behavioral, and it's underrated. The moment developers could see their own per project usage, they self corrected. Nobody wants to be the person burning 5x the team on commit messages. Visibility alone changed behavior before any hard caps kicked in.

Where this led

I ended up building an internal gateway that sits between our developers and the model providers, so every request is attributed to a person and a project, checked against a budget, and routed to an appropriate model, with caching preserved and hard caps enforced. It eventually became a product (Clawgate), but honestly the approach matters more than the tool. You can assemble most of this yourself with an open source proxy like LiteLLM plus some discipline. The important part is that you stop flying blind.

The result for us was straightforward: same shipping speed, materially lower bill, and, maybe more importantly, I can now answer the question "what is our AI spend buying us?" instead of shrugging at it.

The takeaway

Agentic coding is worth every bit of the hype on the productivity side. But the cost side is the least governed line in most engineering budgets right now, and it's growing fast. You don't need to slow your team down to control it. You need to see it, attribute it, cache aggressively, right size the models, and cap the loops. The gains are real. Just don't let them leak out the bottom as a surprise bill.

How are you all handling AI coding costs on your teams? Curious whether others are seeing the same "invisible until the invoice" pattern, and what's worked for you. Drop it in the comments.

Top comments (1)

Cost controls work best when they are tied to outcome quality, not just token caps. I would track cost per accepted change, retry depth, cache hit rate, and latency by task class. That reveals whether a cheaper route is actually saving money or merely pushing repair work to the next model call.