My routing policy and my traces disagreed 96 times. Never once on the main thread.

This is part four of a series about pointing an append-only audit log at things that count tokens. Part one found that a missing model line was half my agent overspend. Part two found two accounting bugs in splitrail. Part three found a 2.35× over-count in a 30k-star repo that the vendor's own documentation had warned about. I promised this post at the end of part two and pushed it at the start of part three. Here it is. The first three parts asked whether tools counted correctly. This one assumes the counting is right and asks the next question: given trustworthy numbers, do they say what I think they say about my own behaviour? I wrote my routing policy down as a file. Then I measured which model actually ran, per decision, and priced the difference. 425 decisions. 96 deviations, 22.6%, $1,248.13. Zero of them on the main thread. What "stated versus revealed" means here The stated half is a 30-line YAML file. The revealed half is an append-only trace store keyed on message.id . A deviation is a disagreement between them. Three decisions in that file are doing most of the work. It scores tiers, not models. frontier is Opus 4.8, Opus 4.7, Fable 5. mid is Sonnet 5. cheap is Haiku 4.5. Swapping Opus 4.8 for Fable 5 is not a deviation; both are frontier. expected_model exists in the file but never triggers a flag, only appears in the report. An audit that flags every model substitution reports noise, and an audit that reports noise gets switched off. The grain is (unit, component) . A unit is a stretch of my own prompts segmented by idle gap. One unit contains a main-thread stretch plus every subagent it spawned, so each (unit, component) pair gets one verdict and the dominant model wins. One rule in that file exists only because the first draft was wrong. The draft said research work belongs on the cheap tier. Running it produced a pile of main-thread deviations, and they were not deviations. Interactive research with me in the loop on a frontier model is what I actually want; the draft policy was describing a different intent than the one I hold. So main × research-explore → frontier went in, with the reasoning written into the file's own header. That correction is the part I would keep if I had to throw the rest away. A policy you never revise is not a policy, it is a wish. And the revision has to happen before you measure, or every measurement afterwards is scoring your behaviour against a straw man. Why believe the revealed half Before any of the numbers below mean anything, the trace log has to be recording what happened rather than what I meant. Here is the case, and I did not construct it. I found it while looking for something else. Claude Fable 5 became available to me on 2026-06-10. My store's first Fable trace is timestamped 2026-06-10 00:00:38 . Daily Fable traces from there: | date | traces | |---|---| | 2026-06-10 | 1,599 | | 2026-06-11 | 1,541 | | 2026-06-12 | 769 | | 2026-06-13 | 31 | | 2026-06-14 → 2026-07-01 | 0 | | 2026-07-02 | 2,334 | Eighteen days of nothing, then it comes back. The 31 are the interesting part. All of them fall between 00:00:12 and 01:39:07 , the first hundred minutes of that day. They belong to three sessions, and every one of those sessions had started earlier: 2026-06-12 14:57, 2026-06-10 00:00:38, 2026-06-10 11:28. Not one new Fable session began on 2026-06-13. That shape is hard to produce by choosing. If I had decided to stop using Fable, the last day would contain new sessions that ended early, because deciding to stop is something I do between tasks. What the log holds instead is drainage: work already in flight, finishing. A decision stops new work. An availability change stops new work and lets running work drain. I never told the trace store about any of this. It has no concept of a model being available. It records model_id per message because that is the field that arrives, and by recording it faithfully for a boring reason it captured an external event nobody instrumented. That is the whole argument for the revealed half, and it is the strongest form the argument can take: the log is credible about my intentions precisely because it does not know what my intentions were. The result Where the deviations are. | component | decisions | deviations | deviation cost | |---|---|---|---| main | 314 | 0 | $0.00 | Explore | 14 | 0 | $0.00 | workflow-subagent | 73 | 73 | $1,146.23 | general-purpose | 23 | 23 | $101.90 | product-manager | 1 | 0 | $0.00 | | total | 425 | 96 (22.6%) | $1,248.13 | 95 of the 96 are mid → frontier , $1,248.10. The last one is mid → cheap and costs three cents. So this is not scatter in both directions. It is one direction, in one place. Those numbers are a snapshot of one window, 2026-05-30 to 2026-07-04, and I am going to hold myself to the standard I apply to everyone else in this series: they reproduce from a frozen copy of the database, not from the live one. I have since backfilled tags over the following five weeks and regenerated, which gives 784 decisions and 161 deviations at 20.5%. That is not a corrected version of the table above and I am not going to present it as one. The denominator changed, and every unit added carries an automatically assigned task type that no human has reviewed. Two numbers computed over different populations are two numbers. Every deviation is a subagent. Every main-thread decision is compliant. The main thread is where I sit, where I pick a model on purpose, where I notice what I am spending. It never once got it wrong. What the subagents were running. For each deviation, I asked what the parent session's main thread was running inside that unit's own time window. | verdict | n | cost | |---|---|---| | main ran exclusively the subagent's model | 95 | $1,248.10 | | main never on the subagent's model | 1 | $0.03 | | main mixed, or untestable | 0 | $0.00 | Before that table means anything: 312 of 314 unit windows have a single-model main thread anyway. "Main was exclusively on one model" is nearly free. The comparison that carries the weight is against the subagents that did not deviate. | subagent decisions | ran the parent's model | ran something else | |---|---|---| | deviating (96) | 95 | 1 | | compliant (15) | 1 | 14 | 95 of 96 against 1 of 15. And the 14 compliant non-matches are all Explore on Haiku, which is the component whose agent definition pins model . Pinned subagents ignore the main thread. Unpinned ones don't. The mechanism is documented upstream: a subagent whose definition omits model takes the main thread's. That it produced these 95 is inference, and I want to keep those two sentences next to each other. What the contrast rules out is deliberate per-subagent selection. Choosing on purpose 95 times would leave mixed windows, and there are none. The one compliant match is the most useful row in the table. It is product-manager on Opus 4.8, $0.91. No rule in my policy covers that component, so it fell through to default_tier: frontier and scored compliant. It inherited exactly like the other 95. The policy's own default hid it. A component with no rule cannot deviate. Which means gaps in a stated policy read as compliance, and the more incomplete your policy is, the better your compliance looks. That is a warning about this method, produced by the method, and it is worth more to me than another confirming number. I got that table wrong the first time. The first run said 75 exclusive matches, 20 untestable, 1 non-match. traces.invoked_at is declared DATETIME , which gives the column NUMERIC affinity in SQLite; my open-ended upper bound '9999' converted to the integer 9999, and every integer sorts before every text value, so the comparison was false for all twenty rows with an open window. Both runs summed to 96 rows and $1,248.13. The cross-foot reconciled and the classification was still wrong. What it cost, and what the fix is worth $1,248.10 of $1,248.13 sits in the match bucket. The config-field hole is not most of the deviation cost, it is all of it but three cents. Part one already called a missing model line half my overspend. That was an anecdote with a number attached. This is the same claim with a control group behind it, and the interesting part is not the size. It is the location. Deviation clustered perfectly on the components where no human is choosing. I built this expecting to find undisciplined routing, and discipline turned out not to be the failing variable. It is a way of finding the places in a system where no decision is being made at all, and pricing them. The 50% that is arithmetic, not a finding 79 Fable units, re-priced against Opus 4.8 with the token mix held fixed, saves $1,383.14, exactly 50.0%. That number is a corollary of the price sheet and nothing else. Same tokenizer, same token counts, and Opus is half the price of Fable, so the saving is identically the price ratio. It could not have come out any other way, and quoting it without this paragraph attached would be quoting a division problem as a result. The two questions that would make it a finding are whether quality held, and whether the token count itself changes on a different model. Neither is answered by arithmetic. One of them is answered badly below. The blind eval, and three ways it is compromised 12 pairs, judged blind, original answer against a cold replay on the cheaper model. The main result uses only the 8 pairs where the reviewer did not recognise the original: original wins 7, replay 1, tie 0. Read that asymmetrically. Replay wins-or-ties 1 of 8. That is the direction that would support downgrading, and it is weak. This is not evidence for downgrading. Original wins 7 of 8. That cannot be read as the Fable premium being justified. The replay is a cold start without the original conversation's context, so its disadvantage may be missing context rather than model quality. This design does not separate them. Three flaws, all mine: The reviewer recognised 4 of 12. The reviewer was me, judging my