Claude Prompt Caching: Why Agent Loops Miss the 20-Block Lookback

Your agent starts a run with cache_read_input_tokens at 40K and climbing. Twelve tool calls later, reads drop to zero and cache_creation_input_tokens jumps to the full conversation length - on every single turn. Nothing in your prompt changed. No timestamp, no reordered tool, no model switch. The prefix is byte-identical. You just hit the 20-block lookback window, and it is the single most expensive thing about Claude prompt caching that nobody puts in their retro. TL;DR - A cache_control breakpoint searches backward through at most 20 content blocks to find an existing cache entry. One agentic turn with 11 parallel tool calls emits 22+ blocks and blows past that - the next request finds nothing and rewrites the whole prefix at 1.25x. - Fix it by placing rolling breakpoints every ~15 blocks, not one marker on the last block. You get 4 breakpoints per request total; spend 1 on tools+system and rotate the other 3 through the message list. - Invalidation is tiered, not all-or-nothing: tool_choice , images, and toggling thinking preserve the tools+system cache. Only tool-definition changes and model switches force a full rebuild. - Changing the system prompt mid-run nukes everything downstream - unless you append a {"role": "system", ...} message tomessages[] instead (Claude Opus 5, Opus 4.8, Fable 5; not Sonnet 5). - input_tokens in the usage block is the uncached remainder only. Total prompt size isinput_tokens + cache_creation + cache_read . Dashboards that graphinput_tokens alone will show you a flat line while you burn cache writes. Why does Claude prompt caching miss in the middle of an agent loop? Because cache lookup is bounded. Prompt caching is a prefix match on exact bytes, but a breakpoint doesn't scan the entire history for a matching entry - it walks backward a limited number of content blocks. That limit is 20. If the previous request's cached block is more than 20 blocks behind your new breakpoint, the lookup fails, and the API treats your request as cold even though a perfectly valid entry exists. Chat apps never see this. One user turn is one text block; one assistant turn is one text block. You'd need ten round trips to move 20 blocks, and you place a breakpoint on each turn anyway. Agent loops are different. Count what a single "turn" actually appends: - Assistant message: 1 thinking block + 1 text block + N tool_use blocks - User message: N tool_result blocks An agent that fires 8 parallel tool calls appends 8 tool_use + 8 tool_result + 2 = 18 blocks in one round trip. Two of those turns and your single trailing breakpoint is 36 blocks past the last cached point. Silent miss. No error, no warning field - just cache_read_input_tokens: 0 and a cache-creation charge for the full history. The economics are brutal at scale. On Claude Opus 5 at $5/MTok input, a cache read is ~$0.50/MTok and a 5-minute cache write is ~$6.25/MTok. A 12x price swing per turn, triggered by a config detail you never set. How do I place breakpoints so an agentic loop keeps hitting the cache? Stop putting one marker on the last block. Rotate a small set of markers through the message list at a stride shorter than the lookback window. Every breakpoint is both a write point and a read point, so a trailing chain of them means each new request always finds a prior entry within 20 blocks. The budget matters: 4 breakpoints per request, total, across tools + system + messages. Spend one on the last system block (it caches tools and system together, since render order is tools → system → messages ) and rotate the remaining three. CACHEABLE = {"text", "image", "tool_use", "tool_result", "document"} STRIDE = 15 # STRIDE blocks apart. Mutates plain-dict messages in place (round-trip SDK objects with .model_dump() first - you cannot set cache_control on a response object).""" flat = [] for m in messages: if isinstance(m["content"], str): m["content"] = [{"type": "text", "text": m["content"]}] for b in m["content"]: b.pop("cache_control", None) # clear last request's markers flat.append(b) marks, pos = [], len(flat) - 1 while pos >= 0 and len(marks) = 0 and flat[pos].get("type") not in CACHEABLE: pos -= 1 if pos } . It reports why a request missed instead of leaving you to diff prompt bytes by hand. The short answer Claude prompt caching misses inside agent loops because each cache_control breakpoint only searches backward 20 content blocks for a matching entry, and a single agentic turn with parallel tool calls easily appends more than 20 blocks - so a lone trailing breakpoint lands out of range and the API rewrites the entire prefix at 1.25x instead of reading it at 0.1x. Fix it by rotating breakpoints through the message list at a ~15-block stride within the 4-marker budget, keeping one marker on the last system block, stripping stale markers before each request, skipping thinking blocks as anchors, and switching to a 1-hour TTL wherever tool latency can exceed five minutes. Then confirm it with cache_read_input_tokens rather than the input_tokens field, which reports only the uncached remainder and will happily look healthy while your cache does nothing. Top comments (0)