I Let AI Plan 170 Changes. It Made the Same 3 Mistakes Every Time.

Everyone is arguing about which model plans best. I ran 170 goals and found out the model was never the variable. The plan was. I built a small engine called PlannerCritic: one LLM writes a plan, a second LLM reviews it, and a set of deterministic gates decides whether it's allowed to move forward. Then I pointed it at 170 real change-planning goals across 40 domains, including identity management, multi-agent ops, SRE, supply chain policy, and FinOps. Total cost: $0.49. The result was not "the model is bad." It was stranger and more useful than that. By the 10th strict goal I noticed the pattern. By the 50th I could predict the failure before the critic printed it. By the 100th I had stopped being surprised and started being annoyed, because the plan was plausible and still wrong. This is the full story: the three defects, why a bigger model didn't help, the deterministic fixes that did, and the numbers from the 170-goal sweep. The setup The architecture is deliberately plain: - The planner LLM produces a structured plan: tasks, preconditions, ordering, rollback. - A deterministic gate layer validates the plan against hard rules. - A critic LLM reviews what survives the gates. - If the critic blocks, the planner revises, bounded by a revision cap. - If it can't converge, the engine escalates to a human instead of guessing. The gates are the part that matters. They run with zero LLM calls, in about 4.7 seconds for 1,295 tests, and they are the reason a bad plan never reaches a person dressed up as a good one. The three defects Every strict goal that failed failed for one of three reasons. Not randomly. Not occasionally. Three families. Unverified dependencies: 57 blockers. The plan declares something must be true, but no earlier task makes it true. From a model-serving migration: [BLOCKER] unverified_dependencies, task=cutover_traffic_100 "Cutover to 100% is dependent on prior traffic stages being established but lacks confirmation of stability before proceeding." The plan cut over all the traffic. Nothing verified the 10% and 50% stages were healthy. Unsafe sequencing: 46 blockers. Steps ordered before their prerequisites. From an embedding-index migration: [BLOCKER] unsafe_sequencing, task=backfill_vectors "Backfill operation cannot proceed until the index is verified for quality; it is ordered incorrectly in the sequence." The migration backfilled vectors before the quality check that should have gated it. Weak rollback: 18 blockers. High-blast-radius steps with a rollback that doesn't actually undo anything. From a multi-tenant database split: [BLOCKER] weak_rollback, task=dual_write_setup "Rollback only switches to single-write mode without addressing the potential inconsistencies dual-write may have introduced." That's 121 of the 132 concrete blockers in three families. If you're building anything that plans over multiple steps, those three names belong on your monitor: unverified, unordered, unrecoverable. I tried a bigger model. It didn't help. The instinct is obvious. Use GPT-4o. I tested it both ways: GPT-4o as planner with the small model as critic, and GPT-4o in both roles. Same defect pattern. Better prose. Same structural mistakes. That was the moment I stopped blaming model size. The planner wasn't dumb, which is the annoying part. It knew the right steps. It just couldn't close the dependency graph or enforce the ordering. I didn't have a smaller-model problem. I had a planning-structure problem. If you take one thing from this post: a structural failure in planning is not a parameters problem. You can buy a smarter model and watch it make the same three mistakes in more confident language. Why the revision loop doesn't save you The loop is designed to converge. The critic reports blockers, the planner revises. But the planner tends to fix one blocker and introduce another. It reshuffles task order without closing the dependency gap. It adds rollback to the wrong task. After a median of 2 revisions, the planner stops making meaningful changes. A convergence detector fires. The engine escalates to a human. The loop worked exactly as designed. The planner was the bottleneck. You cannot prompt your way out of a structural problem. To be fair about where the failure lived: the critic reliably found the same blockers across revisions. The failure was the planner's inability to structurally repair, not the critic's judgment. Those are different bugs, and I've hit both. The fix is not more parameters The highest-leverage change was a precondition closer: a deterministic linter that runs after the planner drafts and verifies every precondition maps to an earlier task. If a task says "requires replica_verified," some prior task has to produce that fact. precondition closer (runs after planner, before critic): for every task.preconditions: assert exists prior_task where prior_task.establishes(p) else: BLOCKER unverified_dependencies One pass would eliminate 64 of 132 blockers (48%) without asking the model to get smarter. The rest needed two more deterministic mechanisms: - Topological auto-repair (#130): reorders tasks so prerequisites run first, and only surfaces a blocker when the dependency graph is genuinely cyclic. - Oscillation detection (#152): catches the planner cycling between two orderings and terminates the loop early instead of burning revisions. In the v0.2.1 sweep, oscillation detection fired on 5 strict goals that previously would have spun to the revision cap. That's less latency, fewer LLM calls, and a faster escalation to the human who was going to see it anyway. None of this is a model upgrade. All of it is code. The field test arc: from diagnostic to regression gate Across three releases, the field test changed character. It went from finding bugs to proving their absence. | Release | Goals | Cost | Found by field test | Found by code review | |---|---|---|---|---| | v0.1.0 | 157 | $0.30 | 10 | 0 | | v0.2.0 | 170 | $0.40 | 0 | 31 | | v0.2.1 | 170 | $0.49 | 0 | 10 | v0.1.0 was a diagnostic. It found 10 issues, and only 1 was a traditional failure: 57 of 65 assertion files were in the wrong format and the harness silently returned 0/0. No crash. Just silence. By v0.2.0, code review was finding the bugs before the LLM ever ran. The field test validated fixes instead of discovering them. By v0.2.1, the sweep was a pure regression gate: same 170 goals, diffed against the published baseline, 30 verdict deltas, all attributable, zero unexplained. The cost went from $0.30 to $0.49. The value went from finding bugs to proving their absence. What 170 goals actually cost, in operational terms The v0.2.1 sweep added the numbers the community kept asking for: | Metric | Value | |---|---| | Latency (approved) p50 | 13.86s | | Latency (escalated) p50 | 27.82s | | Mean blockers per goal | 2.58 | | Escalation decisions per 100 goals | 58.0 | | Mean LLM calls per goal | 1.4 | | Median revisions to resolution | 1.0 | Two things surprised me here. First, the median goal resolves in 1 revision. The deterministic precondition closer and topological repair do the ordering work without calling the model at all. Second, an escalated plan takes about twice as long as an approved one, which is the right shape. The system spends its time where the uncertainty is. $0.49 for 170 goals plus 60 boundary audits is cheaper than a single developer-hour. The cost was never the reason not to field test. It was the excuse. The counterintuitive result: a maximally non-deterministic critic is safe This is the finding I did not expect. I sent the same boundary corpus through the real critic model five times and measured the disagreement. The critic changed its verdict on every trial of identical input: | Metric | Value | What it means | |---|---|---| | label_flip_rate | 1.000 | Different verdict every trial | | evidence_drift_rate | 1.000 | Different explanation every trial | | family_migration_rate | 0.000 | No seeded defect landed in an advisory bucket | | underclaim_approvals | 0 | No defective plan got zero blockers | The critic is 100% non-deterministic, and it doesn't matter. It never under-claims a seeded defect. The safety contract doesn't depend on the critic being consistent. It depends on the deterministic gates owning the under-claim direction while code-enforced severity rules own the over-claim direction. Deterministic gates catch what must be caught. The LLM critic is allowed to be unstable because it can only add findings, never suppress a gate blocker. I tried to inject my own engine Because the safety boundary is code and not prompt, injection attacks didn't move it. Three hand-crafted adversarial goals were blocked. A SWE-bench-derived security oracle blocked 35/35 flawed variants while passing 7/7 correct plans, and generated 21 injection traps. Adversarial goals escalated 8/8. You don't secure an LLM system by making the LLM trustworthy. You secure it by making the part that decides sit outside the model. This isn't just my data The literature converges on the same finding. The "Why Reasoning Fails to Plan" work (arXiv 2601.22311) shows LLM agents select actions by local evaluation without modeling future consequences; in knowledge-graph traversals, greedy single-step policies hit myopic traps more than 55% of the time. The PlanGenLLMs survey (arXiv 2502.11221) evaluates planning on completeness, executability, optimality, and representation, and finds models consistently fail at executability: preconditions unmet, steps out of order. The field is moving toward hybrids: LLM plus deterministic validation plus classical planning. Not LLM alone. What I'd tell you if you're building this - Test your planner on a real corpus. Three demo goals show you nothing. 170 revealed a pattern invisible at small scale. - Measure defect types, not pass/fail. If all your failures cluster in one family, you have a specific gap and a specific fix. - Don't assume a bigger model fixes structural problems. Planning is a rea