Survey Free-Text Classification: Schema, Confidence, and Re-Run Design

Free-text survey and inquiry responses do not classify themselves, and a single "summarize this" prompt does not produce anything you can act on twice. What you need is a schema, a confidence threshold, and a re-run rule.

The Minimum Schema

Three fields, added to whatever the original submission already has:

• category - one of a fixed, small set (pricing, onboarding, support, bug, other)
• sentiment - positive | neutral | negative
• confidence - 0.0–1.0, how sure the classification is

Keep category closed. An open vocabulary just turns into a second free-text problem with extra steps.

Confidence Is a Routing Decision, Not a Footnote

A confidence score only earns its place if something happens differently below a threshold.

if (row.confidence < 0.6) {
  row.status = "needs_human_review";
} else {
  row.status = "classified";
}

Below threshold, the row waits for a person instead of getting auto-tagged, auto-routed to an owner, or auto-counted into a theme total. A model that is unsure and a model that is confident should never look the same in the data. Silent low-confidence auto-action is how a pipeline quietly poisons a report weeks later, once nobody remembers which rows were guesses.

Re-Runs Have to Be Idempotent

You will re-classify the same batch more than once - a prompt improves, a category gets added, someone reruns last week's data by mistake. If the re-run is not idempotent, tags duplicate and downstream actions, like a Slack alert or an owner assignment, fire twice for the same response.

const key = `${response_id}:${classifier_version}`;
if (alreadyClassified.has(key)) skip();
else { classify(response_id); record(key); }

Version the classifier, not just the response, so a genuine re-classification stays a deliberate, traceable event rather than a side effect of re-running a script.

Where the Inference Actually Runs

None of this needs a server-side LLM. Run against FORMLOVA response data, this classification happens in your own connected AI client - during an MCP session, using your own model - not inside FORMLOVA's infrastructure. FORMLOVA exposes the structured response data and stores the resulting fields; it does not classify your responses for you on a server.

The full context for voice-of-customer analysis, including which sources to mix and how to report findings, is here: Voice of Customer Analysis with AI.