My AI agent failed obvious tasks, and 49% fewer retrieval misses changed how I debugged it

I used to blame the model. If an agent missed a refund rule, forgot a tool result from 10 seconds ago, or grabbed the wrong SKU from docs, I'd assume GPT-5 or Claude had a reasoning problem. I don't think that anymore. A lot of "agent is dumb" bugs are retrieval bugs.

That sounds obvious in hindsight, but it changes how you debug everything: n8n flows, OpenAI File Search, support bots, internal copilots, and custom agent stacks held together with Redis, Postgres, and optimism. Anthropic's Contextual Retrieval writeup put hard numbers on something a lot of us have seen in production:

• 49% fewer failed retrievals with Contextual Retrieval
• 67% fewer failed retrievals when reranking is added

That is not a small lift. That is a giant sign that many agent failures happen before the model even starts reasoning. The failure looked like reasoning. It wasn't.

The pattern I kept seeing

The agent could:

• summarize a long PDF
• call an API correctly
• produce a decent customer reply
• follow a multi-step workflow

Then it would fail one painfully obvious step:

• miss the refund window in the policy doc
• use the wrong product SKU
• forget a prior tool result
• ignore a customer-specific exception

When that happens, it feels like bad reasoning. But usually one exact fact was missing at one exact moment. That's not a reasoning failure. That's failed fetch.

The debugging mistake: treating all memory as one thing

A lot of teams say "memory" like it's one subsystem. It isn't. In practice, you usually have at least 3 different layers:

Memory type	Scope	Best for
Session/chat memory	Current conversation or run	Short-term continuity
Durable memory	Across runs, users, or sessions	Preferences, state, long-lived facts
Retrieval	Pulling external facts on demand	Docs, policies, tool outputs, exact references

If your n8n agent forgets a tool result from the same run, that's probably session memory. If it loses a customer preference from yesterday, that's durable memory. If it can't find the refund rule that definitely exists in your docs, that's retrieval. Different bug. Different fix.

This is why debugging gets weird when people throw Redis, Postgres, vector search, chat history, and tool outputs into one bucket called "memory."

Long context does not magically fix retrieval

I still hear this one a lot: "We gave the model the docs, so retrieval can't be the issue." Not true. The Lost in the Middle result is still one of the most annoying realities in production: models often do worse when the relevant info is buried in the middle of a long prompt.

So you can have both of these problems:

• The agent never retrieves the right fact
• The agent retrieves it, then buries it where the model is less likely to use it

That means "the info was technically present" is not a useful defense. If the right fact is hidden in a wall of context, your agent can still fail in a way that looks like reasoning.

Pure vector search loses stupid fights

This is where I'll be blunt. If your agent needs exact strings, pure embedding search is not enough. I'm talking about:

• order IDs
• policy titles
• product SKUs
• workflow names
• error codes
• ticket IDs

Semantic retrieval is great until you need literal precision. That's why hybrid retrieval keeps winning in real systems. Keyword search + semantic search + reranking is just more reliable than hoping embeddings will infer everything. Even OpenAI's retrieval stack leans this way. That should tell you something.

A practical OpenAI File Search setup

If you're using OpenAI-compatible tooling, retrieval should be in the agent loop instead of relying on the model to remember everything from prior turns.

Example:

from openai import OpenAI client = OpenAI () vector_store = client . vector_stores . create ( name = " Support FAQ " ) client . vector_stores . files . upload_and_poll ( vector_store_id =