5 Claude Code Patterns for Rock-Solid Structured AI Output

TL;DR

For months I had an autonomous coding agent that "mostly" worked - until it didn't, because it was answering multi-step questions in free-form prose and I was regex-parsing the answer. Switching every agent decision to schema-validated structured output (forced tool calls, not "please reply in JSON") turned flaky automation into something I can actually trust unattended. Here are the five patterns that made the difference, with code.

The Problem

I run a fully autonomous coding agent that plans its own work, executes multi-step tasks, and decides things like "did this test actually pass," "should I retry this step or escalate," and "is this diff safe to merge." All of those are yes/no-plus-details decisions. Early on, I did the obvious thing: asked the model a question, got back a paragraph, and tried to extract the answer with string matching.

response = ask_agent("Did the test suite pass? Explain your reasoning, then answer yes or no.")
if "yes" in response.lower():
    proceed()

This is fine in a demo. It is not fine in production. Here's what actually happened over a few weeks of unattended runs:

• The model said "the tests did not fail" - my substring check for "fail" fired anyway and killed a good run.
• One response wrapped the verdict in a markdown table. My parser choked.
• Another time the model hedged ("mostly passing, one flaky test") and my binary check had no idea what to do with that.
• Worst one: a response that said "no" in the reasoning paragraph but "yes" in the final line - because the model changed its mind mid-answer and I was reading the wrong sentence.

None of these were model failures. They were interface failures. I was asking a probabilistic text generator to produce output that a brittle string parser had to reverse-engineer. The fix wasn't a better prompt - it was removing free text from the loop entirely.

The incident that finally forced my hand: the agent ran a multi-step refactor, hit a genuinely broken test, and wrote three paragraphs of reasoning that started with "the test is failing because..." My parser matched on the word "failing" appearing anywhere in the response and correctly flagged it - except a different run, on a passing suite, produced a summary that said "no previously failing tests are failing now," and the same substring match flagged that one too. Two runs, opposite outcomes, identical parser behavior. That's when I stopped trying to make the regex smarter and started asking a different question: why was I letting the model choose its own output format at all?

How I Solved It

The core idea: instead of asking the model to write about its decision, force it to call a function whose arguments are the decision. Claude Code and the underlying API support this natively via tool use with a JSON schema - the model doesn't get to choose the shape of the reply, it has to fill in a contract you defined.

Pattern 1 - Force the tool call, don't suggest it

Defining a tool isn't enough on its own; the model can still decide to respond in prose. You have to force it.

import anthropic

client = anthropic.Anthropic()

verdict_schema = {
    "name": "report_verdict",
    "description": "Report whether the test suite passed",
    "input_schema": {
        "type": "object",
        "properties": {
            "passed": {"type": "boolean"},
            "failing_tests": {
                "type": "array",
                "items": {"type": "string"}
            },
            "confidence": {
                "type": "string",
                "enum": ["high", "medium", "low"]
            }
        },
        "required": ["passed", "failing_tests", "confidence"]
    }
}

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    tools=[verdict_schema],
    tool_choice={"type": "tool", "name": "report_verdict"},
    messages=[{"role": "user", "content": f"Here is the test output:\n{test_output}"}]
)

verdict = response.content[0].input
if verdict["passed"]:
    proceed()

tool_choice={"type": "tool", "name": "report_verdict"} is the whole trick. It's not "here's a tool if you want it" - it's "you may only respond by calling this tool." No prose leaks out, no markdown table to parse, no substring matching.

Pattern 2 - Make "I don't know" a valid answer, not a crash

A schema with only passed: boolean quietly punishes the model for being honest about uncertainty - it has to pick true or false even when the real answer is "the log got truncated, I can't tell." So I always add an explicit escape hatch:

{
    "properties": {
        "passed": {"type": ["boolean", "null"]},
        "reason_unknown": {"type": "string"}
    }
}

If passed comes back null, my code branches to a human-escalation path instead of forcing a guess. This single change cut down a category of bugs where the agent would confidently report success on ambiguous input just because the schema didn't leave room for "unclear."

Pattern 3 - Keep the schema flat and small

My first attempt at this had deeply nested objects - arrays of objects containing arrays of enums. It technically worked, but validation failures were common and the model would occasionally emit a subtly malformed structure (an object where an array was expected, three levels deep). Flattening the schema fixed most of it:

// Before - nested, fragile
{
    "steps": [{
        "name": "...",
        "checks": [{"id": "...", "status": "..."}]
    }]
}

// After - flat, robust
{
    "step_names": ["..."],
    "check_ids": ["..."],
    "check_statuses": ["pass", "fail"]
}

Parallel flat arrays are uglier to read in the schema definition, but the model produces valid instances of them far more reliably than deep nesting. If you're seeing intermittent validation errors, flattening is the first thing I'd try before touching the prompt.

Pattern 4 - Validate before you trust, always

Structured output narrows the failure mode, it doesn't eliminate it. I run every tool-call result through a schema validator (jsonschema in Python) before touching it, and treat a validation failure exactly like a model error - retry once with the validation error appended to the context, then escalate.

from jsonschema import validate, ValidationError

try:
    validate(instance=verdict, schema=verdict_schema["input_schema"])
except ValidationError as e:
    verdict = retry_with_error_context(str(e))

This one retry loop caught more real issues than I expected - usually the model self-corrects immediately once you show it the exact validation error instead of just asking again.

Pattern 5 - One decision per call

I used to bundle multiple decisions into a single tool call ("report the verdict AND suggest a fix AND estimate risk") to save round-trips. It seemed efficient. It also meant that when one field was garbage, I had to throw away or manually patch the whole response. Splitting into single-purpose tool calls per decision made retries cheap and isolated - a bad risk_estimate call doesn't force me to redo a perfectly good verdict call.

flowchart LR
A[Agent step output] --> B[report_verdict tool call]
B --> C{Schema valid?}
C -->|yes| D[Branch on structured fields]
C -->|no| E[Retry with error context]
E --> B
D --> F[Next agent step]

Lessons Learned

• Forced tool calls beat "please respond in JSON" every time. Asking nicely for JSON in a prompt still leaves room for the model to wrap it in prose or markdown fences. tool_choice removes the choice entirely - that's the point.
• An escape hatch for uncertainty prevents confidently wrong answers. If your schema can't express "I don't know," the model will pick an answer anyway. Give it a null or an unknown enum value and route it to a human.
• Flat schemas fail less than nested ones. This surprised me - I expected structure quality to be the limiting factor, but shape complexity mattered more than I assumed.
• Validation isn't optional even with structured output. Schema-following models are much better than free text, not perfect. Treat every tool-call result as untrusted until it passes validation.
• Smaller, single-purpose calls are easier to retry than one big call. Bundling decisions to save API round-trips backfires the moment any single field needs a retry.

What's Next

I'm now working on wiring the same validated-output discipline into the agent's planning layer, not just its verdicts - having it commit to a structured task list up front instead of describing a plan in prose and having me infer the steps. Early results are promising but the schema design is trickier when the output is "a variable number of steps" rather than a fixed decision: an array of objects is exactly the nested shape that Pattern 3 warns against, so I'm experimenting with capping the array length and giving each step a fixed, flat set of fields rather than letting the model invent sub-structure per step.

The other thing on my list: schema-versioning. Right now if I change a tool's input_schema, every in-flight retry loop that still has the old schema in its conversation history gets confused. I haven't solved this cleanly yet - the current workaround is just "don't change schemas while agents are mid-run," which is a process fix, not an engineering one. If you've solved schema migration for long-running agent conversations, I'd genuinely like to hear how.

One more pattern worth a mention, even though it didn't make the top five: log the raw tool-call input alongside the validated result, not just the parsed fields. The first few times validation failed, I'd thrown away the raw payload and only kept an error message - which meant I couldn't tell whether the model had produced almost-valid JSON or something wildly off-schema. Once I started logging the raw input every time, debugging validation failures went from guesswork to a five-minute diff.

Wrap-up

If your agent pipeline is still parsing free text with regex or substring checks, that's very likely your biggest reliability bug, not your prompts. Try forcing a single tool call for your riskiest decision this week and see how much flakiness disappears.

If you found this useful, follow me here on Dev.to - I write about building and running autonomous coding agents in production, warts and all.