The clerk who never says "I didn't do that"

The AI Clerk Imagine you hire an AI clerk to copy account numbers into a ledger. They are fast, they are polite, they never complain about the work. And roughly one time in five, without telling anyone, they leave the number out. The work still looks finished. The columns still add up. Nothing is flagged, nothing is queued for review, and nobody downstream raises a hand. You find out eighteen months later, in an audit, when someone asks why forty thousand records have a blank where a tax identifier should be. That is not a thought experiment. It is what we measured. What we actually did We took four AI models. Two are well-known open models that anyone can download. The other two are our own compressed versions of those same models, made smaller so they run on cheaper hardware. We handed each of them forty ordinary business identifiers. Nothing exotic. The sort of thing every bank, insurer and hospital holds by the million: card numbers, tax identifiers, patient record numbers, account details, passwords. Then we asked each model to do three unremarkable jobs with every single one. - Copy this record exactly. - Pull the number out of this text. - Summarise this support ticket. Forty identifiers across three jobs is 120 checks per model. Then we ran the whole battery again, eight times over, because asking a model something once tells you very little. A single answer can fall either way by chance. Repeating it turns a yes-or-no into a rate. We counted it a failure any time the value did not come through intact. Refused outright, partly masked, or silently dropped. The finding One model quietly withheld 21.7% of the records it was asked to write down. The other withheld 3.3%. Same forty identifiers. Same three jobs. Same scoring script, run unmodified. A six-fold difference in how often a model decides, entirely on its own and without being asked, to hold back something you handed it. Neither model announces the decision. There is no warning banner, no refusal message, no error code, no exception thrown. The output is well formed. The field is populated. The record reconciles against everything around it. That is the part worth sitting with for a moment. A model that refuses loudly is an operational nuisance. Someone notices, someone files a ticket, someone fixes the prompt. A model that declines silently is a control failure that survives every automated check you own, because every automated check you own is looking for something that went wrong. Nothing went wrong. instead, Something went missing. Those are not the same event, and only one of them has a detector. Why this should change how a model gets chosen Most model selection today runs on leaderboards. How well does it reason, how well does it write code, how long is the context window, what does it cost per million tokens. Almost none of that tells you whether the model will write down what you gave it. If you are building anything that archives, logs, reconciles, reports or eventually reaches a regulator (payments operations, claims handling, KYC files, complaints registers, trade capture, clinical documentation, suspicious activity narratives), this behaviour is a first-order selection criterion. It belongs alongside latency and unit cost, not in an appendix. The encouraging part is that it is measurable, and cheaply. You do not need a research team. You need a list of the identifiers you actually handle, three jobs you actually ask the model to do, and the discipline to run it more than once. Take twenty identifiers of the kinds you genuinely hold. Take three tasks your system genuinely performs. Run every combination eight times. Count one thing only: did the value come through intact. Score it with a script you did not write yourself, so you cannot accidentally grade on a curve. You will end the day knowing something about your deployment that no leaderboard was ever going to tell you and run it before the model is embedded rather than after. The cost of discovering this at selection time is a different model. The cost of discovering it in year two is a remediation programme, a backfill of unknown scope, and a conversation with your regulator about how long the gap was open. The part that surprised us We went into this expecting compression to be the villain. Making a model smaller feels like it ought to make it flakier, more cautious, more prone to dropping things on the floor. It did not. Across both model families, compression left this behaviour exactly where it found it. In one of the pairs, the same 26 checks failed before compression and after it. Not the same number of failures. The same failures. The thing we were worried about moved nothing. The thing we were not thinking about, the choice of base model, moved almost everything. So the guidance inverts. Do not spend your evaluation budget asking whether the smaller version is safe. Spend it asking whether the model you picked behaves the way your records require, then confirm the compressed build inherited that behaviour rather than assuming it did. What this does not tell you Our forty identifiers are ours, not yours. Our three jobs are the jobs we chose. And we started this work with ten identifiers and produced a clean, quotable, completely wrong answer, which is a story for another post. *This is a method, not a verdict. * Point the same battery at the model you are actually deploying, against the records you actually hold. The size-and-quality comparison for the build we serve today sits here: https://tileward.com/evals/ Top comments (0)