I built a spend cap for LLM calls. It failed by 4.2x under parallel load.

Provider spending limits don't stop anything. They're alerts wearing a brake's clothing. The documented cases from this year are ugly. A developer set a $250 cap and received a $10,138 bill overnight. An AWS customer with anomaly detection enabled was charged $30,141 for a single Bedrock inference run - no alert fired. FinOps teams reported burning an entire annual token budget four months into the year. None of that is because models are expensive. It's structural: provider caps run off billing pipelines that lag by minutes to hours. That was an acceptable design when the worst case was a forgotten EC2 instance at $4/hour. An agent stuck in a retry loop moves faster than the billing system can observe it. So I built a local one. This is the story of getting it wrong first, because the way it failed is more interesting than the fix. Intercepting the calls The first problem is seeing the requests at all. I wanted a wrapper - no code changes for the user: bash burnix --cap 5.00 -- npm run agent The trick is NODE_OPTIONS. When you spawn a child process, you can inject a module that loads before any user code: js const child = spawn(cmd, args, { stdio: 'inherit', env: { ...process.env, NODE_OPTIONS: --require ${hookPath} ${process.env.NODE_OPTIONS ?? ''} , }, }); That hook patches global fetch before the SDK ever captures a reference to it: js const original = globalThis.fetch; globalThis.fetch = async function (input, init) { const url = typeof input === 'string' ? input : input.url; if (!isTrackedHost(url)) return original(input, init); const res = await original(input, init); const clone = res.clone(); const body = await clone.json(); recordCost(body.usage); return res; }; I verified the Anthropic SDK actually goes through globalThis.fetch before building anything else - thirty minutes that would have saved three days if the answer had been no. For streaming, res.body is a ReadableStream you can only consume once, so you have to tee() it: return one branch to the caller inside a reconstructed Response, read the other yourself, and parse the SSE for the final usage event. This all worked. Sequential test: cap of $0.05, blocks on call 3, exits non-zero. Ship it. The bug Then I ran twenty calls in parallel. js await Promise.all(Array.from({ length: 20 }, () => makeCall())); burnix: session 9k5qws done - $0.2100 spent across 20 call(s) Cap was $0.05. Spend was $0.21. Zero calls blocked. The cause is embarrassingly simple once you see it. Cost was added to state after each response returned. Twenty parallel requests all read spent = 0 before any of them completed, so all twenty passed the check: t=0ms req 1..20 all read spent=0, all pass t=800ms req 1..20 all return, each adds its cost t=801ms spent = 0.21. Cap tripped 750ms too late. My sequential test passed. The feature looked done. And the product's single promise - that it stops - was false under precisely the workload it exists for. Agents make parallel tool calls. That is the use case. Reserve, then reconcile The fix is to charge before the call, not after. Reserve - estimate a pessimistic worst-case cost and add it to state immediately, keyed by a reservation id Check - if spent + reserved >= cap, release the reservation and refuse Reconcile - when the response lands, delete the reservation and add the actual cost The estimate is deliberately pessimistic: js const inputTokens = JSON.stringify(body).length / 4; const outputTokens = body.max_tokens; // the API cannot exceed this const worstCase = (inputTokens * inPrice + outputTokens * outPrice) / 1e6; Over-reserving makes the cap trip early, which is the safe direction. Under-reserving is the bug you're fixing. Release the reservation in a finally. A leaked reservation permanently inflates spend for the session, and you will leak one the first time a request throws. The part that actually matters Here's the detail that makes it work, and it's easy to get wrong: The reserve step must be synchronous. readFileSync, mutate, writeFileSync, with no await anywhere between the read and the write. Node is single-threaded. A synchronous read-modify-write cannot be interleaved by another pending promise, because the event loop has no opportunity to run anything else mid-block. The moment you introduce an await between reading state and writing it, you have reopened the exact race you're closing: js // Broken - await creates the interleaving window const state = await readState(); state.reserved[id] = cost; // ← other requests run here await writeState(state); // Correct - no yield point const state = JSON.parse(fs.readFileSync(path, 'utf8')); state.reserved[id] = cost; fs.writeFileSync(tmp, JSON.stringify(state)); fs.renameSync(tmp, path); // atomic swap Reconciliation afterward can be async. The reserve cannot. Result on the same test: burnix: session 4saeq3 done - $0.0454 spent across 6 call(s) Six succeed, fourteen blocked, under the cap. Landing at $0.0454 rather than exactly $0.05 is the pessimistic reservation doing its job - it trips slightly early, which is correct. The bug the fix created Then the display stopped making sense. The progress bar showed settled spend. So you'd watch it climb to 87%, and then: burnix CAP REACHED request blocked Blocked at 87%. To anyone watching, that reads as broken. It wasn't - the reservations had crossed the cap even though settled spend hadn't. But "technically correct" is worthless if the user concludes your tool is lying to them. The fix was to make reservations visible: total $0.0017 / $0.0020 ██████▒▒ 87% (+$0.0005 in flight) Solid blocks are settled spend, shaded blocks are in-flight reservations. Now you watch the combined bar reach the cap and then block, and the behavior explains itself. The lesson generalizes: when internal state drives a user-visible decision, showing only part of that state makes correct behavior look like a bug. What it doesn't do Node children only. The NODE_OPTIONS hook can't reach Python or Go subprocesses. A local proxy would be language-agnostic; that's the next step. Cross-process races are narrowed, not eliminated. Two separate Node processes sharing a session can still interleave between their sync read and write. The window is microseconds instead of seconds. Reservations key off max_tokens. If yours is much larger than your typical response, it trips early. Try it bash npm install -g @burnix/cli burnix --watch -- npm run agent # tracks, blocks nothing burnix --cap 5.00 -- npm run agent # actually stops Works with Groq's free tier, so testing costs nothing. MIT: github.com/pr3tik/burnix One question I haven't found a good answer to: if you run agents on a shared team API key, how do you currently work out who burned the budget? Every answer I've found is "check the dashboard," which tells you the total and nothing else. Top comments (0)