Hindsight Remembered the Defect. SQL Checked the Evidence.

An AI agent remembering a previous manufacturing failure sounds useful. But what happens when it remembers the right symptom and associates it with the wrong root cause?

That was the engineering problem I wanted to explore while building LotLens, a persistent-memory-powered investigation system for manufacturing defects.

The idea behind LotLens is straightforward: when a component fails, an engineer should be able to learn from previous investigations instead of starting from scratch. However, remembering an old incident is only half the problem. The system also needs to know which parts of that memory are supported by actual records.

That distinction ended up shaping almost every important architectural decision in the project.

The problem: similar failures are not necessarily the same failure

Consider a BMS connector that produces intermittent open-circuit readings during vibration testing. An engineer investigating this defect might find several previous cases with exactly the same symptoms. Looking at those cases could save considerable time. But there is a risk: the failures might have entirely different root causes.

For LotLens, I created three synthetic manufacturing cases to explore this problem.

All three cases share the same broad symptom: intermittent open-circuit readings during vibration testing. Yet the available evidence tells us that at least two different causes exist, while the second case remains unresolved.

This is where a conventional search result or a one-shot LLM response can become misleading. Finding a similar incident does not establish that the current incident has the same cause. I wanted LotLens to preserve that distinction.

[Insert screenshot: LotLens dashboard showing all three synthetic cases.]

Why I separated database records from agent memory

The first major decision was to avoid treating the memory system as the primary database. I used a modular monolith with four main technologies:

• React, TypeScript and Vite for the investigation interface.
• FastAPI for the application API and investigation workflow.
• SQLAlchemy with SQLite for local transactional persistence, with PostgreSQL URL support.
• Hindsight for persistent experience retention, retrieval and synthesis.

The important boundary is between SQL and Hindsight. SQL stores the canonical manufacturing records: suppliers, components, production lots, defect cases, corrective actions, QA verifications and audit events. Hindsight stores derived investigation experiences that can be recalled later.

In other words, SQL answers what actually exists in the application. Hindsight helps the system remember which experiences might be relevant. I did not want a recalled memory to independently change a case's status or establish a confirmed root cause.

How I used Hindsight

I integrated the official Hindsight memory system through its Python SDK. The three operations have separate responsibilities:

• Retain stores investigation experiences.
• Recall retrieves potentially relevant past experiences.
• Reflect synthesizes the retrieved information into an investigation narrative.

The application uses asynchronous SDK operations so the memory integration fits into the FastAPI application.

The Recall operation starts with the current defect context. It includes symptoms, component, supplier and production lot. A simplified part of the actual adapter looks like this:

result = await self.client.arecall(
  bank_id = self.bank_id,
  query = query,
  budget = "low",
)
hits = []
for item in result.results:
  text = str(getattr(item, "text", "") or "")
  hits.append(
    MemoryHit(
      text = text,
      memory_type = str(getattr(item, "type", "memory") or "memory"),
      case_codes = sorted(set(CASE_CODE_PATTERN.findall(text))),
    )
  )

The adapter extracts possible case references from recalled memory text. However, extracting a case code does not make that case valid evidence. That required another step.

The important part: validating recalled evidence

This was probably the most important design decision in LotLens. Suppose Hindsight returns a memory that mentions LOT-POC-001. Before using it in the investigation, the application checks whether LOT-POC-001 actually exists in the canonical database.

The investigation endpoint performs that lookup:

recalled_codes = sorted(
  {code for hit in hits for code in hit.case_codes if CODE_RE.match(code)}
)
historical_cases = []
if recalled_codes:
  result = await session.execute(
    case_query().where(DefectCase.case_code.in_(recalled_codes))
  )
  historical_cases = (result.scalars().unique().all())
validated_codes = {item.case_code for item in historical_cases}
unlinked_codes = sorted(set(recalled_codes) - validated_codes)

Only records resolved against the database are included as canonical historical evidence. References that cannot be matched are kept separately as unlinked memory references.

The application also checks whether the Reflect response mentions case codes outside the allowed evidence set and returns citation warnings. This is not full document-level provenance or field-level source validation. It is a case-level validation boundary, but it is an important one.

The next step builds a bounded investigation context using the current canonical case and the validated historical records. The Reflect instructions are deliberately restrictive. They tell the model that:

• Similar symptoms do not prove a shared root cause.
• An ineffective corrective action does not prove a different cause.
• A cause can be described as confirmed only when the canonical record explicitly confirms it.
• Supplier-specific actions should not be generalised without supporting evidence.
• Missing evidence must be reported rather than invented.

I also ask Reflect to separate the current assessment, historical evidence, known and unknown facts, recommended investigation steps, and risks requiring human approval.

[Insert screenshot: Hindsight investigation results showing canonical historical cases and synthesis.]

What happened when I tested it

The first version of the Reflect instructions exposed an interesting failure. When I initially tested the two Supplier A cases, the model incorrectly inferred that because connector reseating and rework had failed in LOT-POC-002, its root cause must be different from the crimp-height issue in LOT-POC-001.

That conclusion was not supported by the evidence. An ineffective corrective action only tells us that the attempted action did not resolve the failure. It does not establish a different cause, and it does not rule out an existing hypothesis.

I revised the evidence constraints and tested the response again. The revised output correctly kept LOT-POC-002 unresolved. It stated that crimp height was neither confirmed nor ruled out for the second batch and suggested checking the actual process measurements and whether the earlier corrective action had been implemented.

I then introduced LOT-POC-003 as a hard negative: the same broad symptom, but a different QA-confirmed cause. The three-case comparison correctly kept the plating issue attached to Supplier B's case rather than transferring it to Supplier A's unresolved investigation.

One captured application investigation returned 21 Hindsight memory hits. That number represents retrieved memory records, not 21 distinct defect cases. The application displayed the canonical historical cases separately.

This was a useful lesson: a plausible-sounding explanation is not the same as an evidence-supported explanation.

QA verification and retaining outcomes

The workflow does not end when Reflect produces a recommendation. An engineer records a corrective action, and the case moves to AWAITING_QA. QA then selects the exact action being evaluated and records its outcome and root-cause status.

The lifecycle rule is intentionally strict:

case.status = (
  "VERIFIED"
  if payload.action_outcome == "effective" and payload.root_cause_status == "confirmed"
  else "INVESTIGATING"
)

An effective action with an unconfirmed cause is not enough to mark the case VERIFIED. Neither is a confirmed cause with an ineffective action.

The verification is committed to SQL before the application attempts to retain the experience in Hindsight. A memory-sync record tracks whether the operation is pending, synced or failed. If Retain fails, the QA verification remains saved. The application records the failure and exposes a manual retry endpoint.

This ordering matters because a temporary memory-service failure should not erase a decision that has already been recorded by QA.

I also wanted unsuccessful corrective actions and unresolved investigations to remain useful historical context. A system that remembers only successful fixes would lose important information about what did not work.

[Insert screenshot: Corrective action and QA verification interface.]

What I learned building LotLens

There were four lessons I found particularly useful.

1. Retrieval and evidence are different things. A memory system can retrieve something relevant without proving that the reference is valid. Checking remembered case IDs against canonical records gives the application a stronger boundary.
2. Negative outcomes are valuable experiences. Knowing that an action was ineffective can prevent engineers from repeatedly trying the same approach. But the system must not turn that failure into an unsupported root-cause diagnosis.
3. Explicit lifecycle rules are better than model decisions. The model can help compare historical cases and suggest the next investigation. The database and QA workflow determine whether a case is actually verified.
4. Persistent memory needs a recovery strategy. The database commit and the Hindsight Retain operation are separate operations. Recording synchronization status and providing a retry mechanism is essential. The current implementation uses a synchronous post-commit attempt and manual retry, rather than a durable background outbox.

Where LotLens stands today

The first investigation vertical slice is implemented with a React frontend, FastAPI backend, SQLAlchemy persistence and Hindsight integration.

The backend test suite has 29 passing automated tests in GitHub Actions. It covers domain validation, API workflows, lifecycle rules, canonical evidence checks, selected Hindsight failure paths and CORS preflight behaviour. I also ran the backend pytest suite successfully in my local Windows environment. The frontend TypeScript and Vite production build passes in CI.

There is still substantial work before this can be considered production-ready. Authentication and organisation-level isolation are not implemented. Database migrations, PostgreSQL integration tests, a durable memory-sync worker, typed recommendation output, field-level evidence provenance and a larger retrieval benchmark remain future work. The current seeded dataset is synthetic and too small to establish general retrieval accuracy.

I also want to improve how the Reflect narrative is rendered in the interface. The current response is Markdown text, so a structured recommendation format would make the results easier to scan and validate.

The project uses Hindsight because persistent experience is central to the workflow, not an optional chat-history feature. The larger idea of agent memory is particularly interesting in manufacturing, where the value of an investigation often extends beyond the batch that originally failed.

My biggest takeaway from building LotLens is that an agent should not simply remember what happened. It should be able to show which records support what it says, preserve what remains unknown, and leave consequential decisions with t That is the direction I want to keep taking LotLens.