If We Can't Verify It, It Doesn't Ship: Engineering an AI Product That Isn't Allowed to Lie

Design rules for an LLM pipeline whose output people act on - from a small product that turns AI job-search research into something you can trust.

Ask any LLM for "20 companies hiring senior PMs in Bengaluru right now" and you'll get 20 names in seconds. Some are hiring. Some froze headcount months ago. One or two may not have the role at all. Here's the trap: the output looks identical either way. Fluency is not evidence, and an invented fact arrives wearing the same confident prose as a checked one.

For most AI products that's an annoyance. For ours it's disqualifying. We sell done-for-you job-search research - scored target roles, company deep-dives, salary bands, a rewritten résumé - and a job seeker acts on it. They spend an evening tailoring an application to a role. If that role closed three weeks ago, the AI didn't save them time; it stole an evening from someone who may be between jobs. One hallucinated fact can cost a real person a real opportunity.

So we adopted one rule and built the pipeline around it: if we can't verify it, it doesn't ship. This post is the engineering version of that rule - the principles that turned out to matter, and what they cost.

Rule 1: LLMs draft, deterministic code decides

The single biggest architectural decision: no LLM in the pipeline is allowed to grade its own homework. Generation is probabilistic; acceptance is not. Every package passes through a verification gate written in plain, boring, deterministic code - date arithmetic, salary-band consistency checks, count assertions, live-URL checks, source-tracing rules.

Why not ask a second LLM to review the first? We do use model-assisted review for semantic questions - but it never gets the final vote. A reviewer model inherits the same failure mode as the generator: it can be confidently wrong, and it can be talked into things. A regex that checks whether a "deadline next month" is actually in the future cannot be sweet-talked. Policy lives in code; models fetch and draft.

The checks themselves are unglamorous, which is rather the point:

• Dates. Every date is checked against the calendar. A deadline in the past fails the gate.
• Salary math. Compensation ranges must be internally consistent - a band that contradicts its own source or its own arithmetic is corrected or cut.
• Live links. Every role link is fetched at build time. Jobs close fast, and we can't stop a posting from dying an hour after delivery - but we never knowingly ship a dead link.
• Provenance. Claims about the candidate must trace to their résumé or their brief. A skill they never mentioned gets flagged for their confirmation - never asserted on their behalf. (This one is non-negotiable: an AI padding your résumé is lying in your name.)

Rule 2: Fail closed, and fail loud

The subtle bug class that scared us most wasn't a failing check - it was a check that silently didn't run. An unreachable validator that returns "no findings" looks exactly like a passing one. A skipped audit is not a pass; it's an unverified package wearing a green badge.

So the gate fails closed: if a check crashes or can't run, the package doesn't move - it goes to a human, with a visible trace of what didn't run. This sounds obvious written down. In practice every "graceful degradation" you add to an AI pipeline is a place where safety can silently evaporate, and you have to hunt them deliberately. Our default changed from "degrade gracefully" to "degrade visibly."

Rule 3: If it can't be traced, downgrade it - never assert it

LLMs fill gaps. It's what they're for, and in research it's exactly what you don't want. Our tracing rule: every claim in a deliverable must be attributable to something actually read - a posting, a filing, a careers page, the candidate's own documents. A claim that can't be traced gets downgraded ("needs confirmation"), cut, or escalated to a human. The one thing it never gets to do is stay in the package as a fact.

The consequence buyers actually notice: our lists are sometimes shorter than a chatbot's. Twelve verified roles beat twenty padded ones, because the eight fillers aren't neutral - each one costs the reader an evening to discover it was filler.

Rule 4: Ship the receipts

Every package ends with a module we call How This Was Built: the searches run, the sources read, and the exclusions made - named, with reasons ("we dropped X because the only posting was 90 days stale"). Two audiences for this:

• The customer, who deserves to see the work, not just the conclusions.
• Us. Receipts are a forcing function. A pipeline that must show its sources can't quietly invent them.

If you're building anything whose output people act on, make the system produce its own audit trail as a first-class artifact - you'll catch more with that than with any prompt that says "do not hallucinate."

Rule 5: A human before send - as long as it's earning its keep

Nothing goes to a customer on autopilot. A person reviews the verified package before it ships. We think of autonomy as something the pipeline has to earn with a measured track record, not something you grant because the demo looked good. The gate catches the mechanical lies; the human catches the ones we haven't written a check for yet - and every one of those becomes the next check.

What this costs (honestly)

Verification is why delivery takes 4 business days instead of 30 seconds, why capacity is limited, and why some claims arrive labeled "needs confirmation" instead of stated smoothly. Slower, smaller, more hedged - every incentive in AI products points the other way, which is exactly why unverified output is what the market defaults to.

And one thing verification doesn't buy: promisable outcomes. We never guarantee interviews, offers, or a salary figure - anyone who does is selling you the confidence, not the work. The guarantee is narrower and real: everything in the package was checked before the customer saw it.

The takeaway, if you're building with LLMs

The interesting engineering in AI products has quietly moved. It's not in the generation - everyone's generation is fluent now. It's in the acceptance path: the deterministic gate, the fail-closed defaults, the provenance rules, the receipts. Fluent is a commodity. Checked is the product.

This is adapted from our verification standard. CareerIntel is a small done-for-you job-search research service - how it works, and a real sample if you want to see what a verified package looks like.