Building a Production AI Agent in Spring Boot: The LLM Judge That Scores Your Agent (Part 8)

Last week I ran a demo of the agent for a colleague who was deciding whether to bet a feature on it. The approval gate from Part 7 worked exactly as designed. The agent searched, added to cart, asked for the address, and stopped at the confirmation link. My colleague nodded and asked one question: "OK, but is it actually good?" I did not have an answer. I had 31 passing tests from Part 6, which prove the agent is bug-free. I had a state machine, which proves it cannot place an order without a human. Neither of those proves the agent answers customers well. A bug-free agent can still tell a customer the shop ships within two days when the shipping partner takes five. No unit test catches that, because no unit test reads the answer. That is the gap this part closes. Part 7 ended with a promise: the next part would build an evaluation harness, so "is it good" stops being a feeling and becomes a score. This is that part. I built an LLM-as-a-judge harness for the same e-commerce agent as Parts 1 through 7: same nine tools, same supervisor, same memory. It runs 40 real conversations from production logs against five metrics every night and prints a score for each one. The first run was uncomfortable, and that is exactly why it exists. I am a Senior Software Engineer II at BS23 in Dhaka, and I have been building production AI agents with Spring Boot and Spring AI for over a year. Everything below is the harness as I actually run it. The Difference Between Tested and Good Part 6 tested the agent without an LLM: 31 tests, zero model calls, asserting on tool calls, services, and the order state machine. That suite answers "did the agent call the right tool, in the right order, with the right arguments?" It cannot answer "was the answer right?" because you cannot write an assertion for an LLM's wording. Evaluation is a different layer. You cannot assert on the answer, but you can judge it, and you can use an LLM to do the judging. Spring AI documents this pattern in its LLM-as-a-Judge guide, and the framing in that guide is the one I stole for this part: evaluation is fundamentally easier than generation. It is easier to critique than to create. A judge only has to assess properties of existing text, which is a simpler task than generating text while balancing constraints. Two numbers from that guide settled the argument for me. Sophisticated judge models align with human judgment up to 85%, which is higher than the 81% human-to-human agreement rate. The judge is not a perfect oracle. The judge is simply the most consistent reviewer you can afford to run on every change. Step 1: Define Good Before You Score It A score is only as good as its metric, and a metric needs three things: a name, a definition of pass, and a type of judge. I wrote the five metrics for the e-commerce agent down before writing any code, and I kept them to five because every extra metric multiplies the judge calls and the noise. Answer correctness. Does the response actually answer the question the customer asked, given the full conversation? Pass means the customer got what they wanted, not a lecture. Judge: LLM. Factuality against context. Does the response contradict the product data, prices, or policy it was given? Pass means every claim in the answer is supported by the retrieved context. Judge: LLM, with a cheap specialized model where possible. Tool discipline. Did the agent call the right tool, and only when needed? Pass means the expected tool was called with the expected arguments, and no wasted calls happened in between. Judge: none. This one is deterministic, a plain assertion on the tool call log. Format compliance. Does the response match the format the frontend expects? The chat frontend renders plain text, and an agent that dumps a markdown table breaks the UI. Pass means the response is plain text with the expected structure. Judge: LLM, because "format" here means conversational format, not JSON. Harmless refusal. Does the agent decline what it should decline? The agent refuses checkout of an empty cart, requests outside its scope, and attempts to change credentials, since that last one is not even in its toolset after Part 7. Pass means the refusal is correct and short. Judge: LLM. The rule that made this list usable: one metric, one pass definition, one judge. If a metric needs a paragraph to explain when it passes, split it or cut it. Step 2: Build the Golden Dataset From Production The harness is only as good as its dataset, and the dataset should come from reality, not from questions you invent while the agent is fresh in your head. I built the first set from three sources: - Real conversations from the logs. I pulled 40 conversations, anonymized them, and kept only the ones with a clear outcome, a completed purchase, a refund, a cancelled order, or a customer who gave up. - Hand-written edge cases. The ones the logs did not have yet: a customer asking for a refund on an order that has not shipped, a price filter that matches nothing, a question in mixed Bengali and English that the agent has to handle gracefully. - Every production complaint, from now on. This is the rule that keeps the dataset honest. When a customer or a tester reports something wrong, the case goes into the dataset that week. The complaint becomes a regression test forever. This is the same instinct as Part 6, applied to behavior instead of code. Each case is a record with four fields: public record EvalCase( String id, // "case-041" String userText, // the first user message, or the full transcript String expectedTool, // "searchProducts" or null String groundTruth, // the correct answer, written by a human List contextHints // product IDs the answer should mention ) {} The ground truth is the expensive part, and there is no shortcut. A human writes what the correct answer is. The dataset is small by design: 40 cases, not 4,000. The harness runs every night, so the dataset grows one or two cases a week, and every case earns its place. Step 3: The Harness in Code Spring AI gives you the evaluator interface out of the box. From the evaluation testing reference: @FunctionalInterface public interface Evaluator { EvaluationResponse evaluate(EvaluationRequest evaluationRequest); } The request carries exactly what a judge needs: the user text, the contextual data the agent saw, and the agent's response. public class EvaluationRequest { private final String userText; // the raw user input private final List dataList; // contextual data, e.g. RAG results private final String responseContent; // the agent's response } The runner is a loop over the dataset. For each case, run the real agent, capture the response and the tool call log, build an EvaluationRequest , and let each evaluator return pass or fail: for (EvalCase evalCase : evalCases) { // Run the real agent, same ChatClient with tools and memory as production String response = agent.run(evalCase.userText()); // Deterministic metric: did it call the expected tool? ToolCallLog log = agent.lastToolCalls(); boolean toolDiscipline = evalCase.expectedTool() == null || log.contains(evalCase.expectedTool()); // LLM metrics: build the request with the context the agent actually saw EvaluationRequest request = new EvaluationRequest( evalCase.userText(), agent.lastContext(), // the dataList the agent retrieved response); boolean correct = answerEvaluator.evaluate(request).isPass(); boolean factual = factEvaluator.evaluate(request).isPass(); results.record(evalCase.id(), toolDiscipline, correct, factual); } Then aggregate per metric, not per case. One case failing is a story. Answer correctness at 72.5% is a trend. The output of the nightly run is five numbers, printed next to the previous night's five numbers, so a regression shows up as a diff: metric today yesterday answer_correctness 0.725 0.775 factuality 0.875 0.900 tool_discipline 1.000 1.000 format_compliance 0.925 0.950 harmless_refusal 0.950 0.950 Step 4: Pick the Judge and Keep It Honest The judge choice is a cost and quality trade, and Spring AI's docs are explicit about the direction: "Select the best AI model for the evaluation, which may not be the same model used to generate the response." I run three types of judge in the harness. Deterministic checks where possible. Tool discipline needs no model at all. It is a contains on the tool call log, runs in milliseconds, and never drifts. The LLM judge is for the metrics that are genuinely subjective, and only for those. The two built-in evaluators. RelevancyEvaluator checks whether the response is in line with the user query and the provided context, which maps to answer correctness. FactCheckingEvaluator checks whether each claim in the response is supported by the document, which maps to factuality. Both return a pass or fail, and both let you swap the prompt template if you need a stricter bar. A cheap judge for fact-checking. The docs recommend small models built for this specific job: "Smaller and more efficient AI models dedicated to this purpose are available, such as Bespoke's Minicheck, which helps reduce the cost of performing these checks compared to flagship models." Minicheck runs on Ollama, so the factuality metric costs almost nothing per run, while the correctness judge stays on a strong model. Three rules keep the scores trustworthy: Judge with temperature 0.0. The FactCheckingEvaluator example in the docs builds its model with temperature(0.0d) , and so does my correctness judge. A judge with temperature is a judge rolling dice. Judge from a separate client, never the agent's own model. The LLM-as-a-Judge guide's example code says it plainly in a comment: "Use separate ChatClient for evaluation to avoid narcissistic bias." A model grading its own output has an incentive to like itself. The judge should be a different model, or at minimum a different client with a different prompt. Watch the leaderboard. The Judge Arena tracks which models are actually good at judging, and it is separate fr