Your JSON Is Valid but Your Data Is Wrong: Five Failure Modes LLM Structured Outputs Won't Catch

Your JSON Is Valid but Your Data Is Wrong: Five Failure Modes LLM Structured Outputs Won't Catch Five failure modes that survive constrained decoding, and why your schema validator will never catch them. Constrained decoding solved a real problem. Before grammar-based methods like Outlines and SGLang, getting valid JSON from a language model was a retry loop. You prompted, parsed, caught the trailing comma, reprompted. Constrained decoding ended that: force token selection through a finite-state machine, and every output parses. Teams adopted it fast. Schema compliance hit near 100%. And then a quiet assumption crept into production codebases: if the JSON validates against the schema, the data is correct. BAML's benchmarks say otherwise. On function-calling tasks, unconstrained generation with post-hoc parsing reached 93.63% accuracy; constrained decoding on the same model scored 91.37%. The always-valid JSON was less accurate than the sometimes-broken JSON. I started tracking this after a classification pipeline I built began returning plausible but fabricated values on roughly one in twelve runs. The JSON always parsed. Pydantic never complained. It took weeks to notice, because every downstream check was structural. Five failure modes keep showing up. They all produce schema-valid output that breaks your pipeline silently: Enum hallucination: valid enum, wrong meaning Confident fabrication: plausible values in free-text fields Cross-field contradiction: fields valid individually, impossible together Distributional collapse: convergence on safe defaults Array hallucination: fabricated entries instead of empty arrays Constrained Decoding: The Problem It Actually Solved Structured output used to mean hoping the model behaved. Constrained decoding was built to fix that, and it did, just not the whole problem. The progression was real. Prompt-and-pray JSON, where you appended "respond in JSON format" and crossed your fingers, gave way to regex-guided generation (LMQL), then to grammar-based constrained decoding. XGrammar, now the default backend for vLLM and TensorRT-LLM, adds near-zero overhead per token. The syntax problem is solved. But solving syntax created a blind spot. Schema validation checks whether a field is typed correctly: a string is a string, a number is a number. It says nothing about whether that string or number is correct. A lock on a filing cabinet keeps the drawers organized. It says nothing about whether the papers inside are accurate. Schema validation works the same way. The reason this matters: forcing a model into a strict output format costs it something. It has to spend part of its attention on staying inside the format, instead of spending all of it on getting the actual answer right. Lee et al. measured that cost directly. Across open-weight models, forcing structured output formats produced a 3-to-9 percentage point accuracy drop. On math reasoning tasks specifically, where getting the reasoning right matters more than the format, the gap exceeded 15 percentage points. Tam et al. found the same pattern from a different angle: the stricter the formatting rules, the worse the reasoning got. The format is not free, and most teams aren't accounting for that cost. Five Failure Modes: What Survives Your Schema Schema validation catches type errors. It does not catch these five failure modes, because each one produces output that is structurally valid and substantively wrong. Enum hallucination. The model picks a valid enum value that is semantically wrong for the input. Consider a priority enum of ["low", "normal", "high", "urgent"] : the grammar guarantees one of those four values, but it does not weight them by input context. The model can return "urgent" on a routine request or "low" on a critical one, and the schema will accept both. Confident fabrication. Free-text fields return plausible but invented data. BAML demonstrated this by submitting a photo of an elephant as a receipt: constrained decoding returned a complete, schema-valid expense report instead of refusing. Constrained decoding eliminates the model's ability to refuse or express uncertainty. The schema requires a value; the model provides one, whether or not the input supports it. Cross-field contradiction. Schema validation checks each field in isolation. It never checks whether the fields agree with each other. A sentiment extractor can return {"sentiment": "positive", "score": 0.1} , a positive label with a score close to zero, which should mean negative. A date parser can return {"start": "2026-03-15", "end": "2026-03-10"} , an end date before the start date. Both outputs pass every individual field's validation. Neither makes sense once you look at the record as a whole, and no single-field validator is built to catch that, because the constraint lives between fields, not inside any one of them. Distributional collapse. The model converges on safe, generic values across different inputs. Constrained decoding biases toward high-probability tokens within the valid set, and "safe" defaults (0.95, "medium", "general") carry higher base probability than context-specific values. I caught this when confidence scores in a classification pipeline flatlined at 0.98 for three weeks. Collin Wilkins documents a similar case where confidence was 0.99 on every output, including gibberish. Every record had valid types, correct enums, reasonable numbers. The distribution had stopped moving, and nothing alarmed because each individual output was structurally correct. Array hallucination. Models resist returning empty arrays. Under constrained decoding, [] is a low-probability token sequence because the grammar weights object-producing paths more heavily than the empty-array path. When a schema requires an items field of type array, the model fabricates entries rather than returning nothing. In extraction tasks, this produces phantom results: your pipeline reports "found 3 matches" when the correct answer is zero. Failure Mode | Signal | Root Cause | Detection | Enum hallucination | Value distribution skew | Grammar selects a valid but contextually wrong token | Track per-field value distributions over time | Confident fabrication | No refusals or nulls | Schema forces a value; model complies regardless | Audit outputs from ambiguous inputs | Cross-field contradiction | Downstream rule failures | Validators scope per-field, not per-record | Pydantic model validators with cross-field logic | Distributional collapse | Field entropy drop | Model defaults to high-probability safe tokens | Monitor entropy; alert on distribution narrowing | Array hallucination | Zero empty arrays | Model treats [] as low-probability under grammar | Track empty-array rate against expected base rate | Five failure modes mapped to their observable signals, root causes, and detection strategies. The Validation Trap: Why More Rules Won't Fix This The first instinct is to write more validation rules. For known patterns, that works. A Pydantic model_validator catches start_date > end_date . A custom check flags sentiment-score mismatches. You can build cross-field constraints for every failure you've already seen. The problem is the failures you haven't seen. Structural correctness is closed: you can enumerate every valid JSON shape for a given schema. Semantic correctness is open-ended. You can't write a rule for a wrong answer you haven't encountered yet. In production, the model finds new ways to be wrong faster than you write validators, and each new validator only covers the last bug. There's a contrarian argument for resampling over constraining that deserves weight here. Instead of forcing the model's output into a grammar as it generates, let it write freely, then check the result: a parser validates the output against the schema, and if it fails, the model simply generates again. This is resampling. Free-form generation lets the model reason without format pressure; the format check happens after, not during. BAML's benchmarks show parse-and-retry outperforming constrained decoding by over 2 percentage points on the same model. You trade guaranteed first-pass parse success for higher accuracy when the output does parse. Whether that trade-off holds at high volume, where retries compound latency, is still an open question. But the deeper problem cuts across both approaches. Structured output hides uncertainty. When the schema requires a value, the model fills it in. A risk_score field always gets a number, even when the model has no basis for the assessment. A summary field always gets text, even when the input contains nothing to summarize. There is no standard mechanism for the model to express "I do not know" or "this field does not apply to this input." The schema is a forcing function, and wrong answers emerge with the same confidence as right ones. After the third time I caught schema-valid-but-wrong output in production, I stopped treating schema validation as a quality gate and started layering semantic checks on top. Schema compliance is the floor, not the ceiling. Three-Layer Defense: Schema, Semantics, Uncertainty Layer 1: Schema and structural validation. This is what you already have: Pydantic, JSON Schema, Zod. It catches type errors, missing fields, and syntactically invalid enum values. Keep it. It solves the syntax problem well. Layer 2: Semantic validators. Cross-field constraint functions that encode business logic: "if sentiment is positive, score must exceed 0.5." Distribution monitors that track field-value entropy over time. When entropy drops below a threshold, you catch distributional collapse before downstream metrics drift. Periodic sample audits on outputs from ambiguous or edge-case inputs catch confident fabrication. This layer requires domain knowledge and ongoing maintenance, but it covers most of the five failure modes, because it checks meaning, not shape. Layer 3: Uncertainty surfacing. Add an optional confidence field a