Your Memory API Is Lying to Your Agent
DEV Community

Your Memory API Is Lying to Your Agent

The memory store may know the truth. The interface may be throwing it away. This piece grew out of a conversation on Edward Izgorodin's post Agent Memory: Everything It Remembers Has the Same Authority, and That Is the Bug. Several of the sharpest points below have names attached, and I have tried to attach them. Imagine an AI agent asks its memory system a straightforward question: What database does the production application use? The memory API returns: [ {"content": "The production database is PostgreSQL.", "score": 0.94}, {"content": "The production database is MongoDB.", "score": 0.91} ] Retrieval worked. It found two highly relevant memories, scored them, ranked them, and returned them. The agent picks PostgreSQL. The production application migrated to MongoDB four months ago. Nothing failed in retrieval. The PostgreSQL record may genuinely be more semantically similar to the query. But semantic relevance was never the question the agent needed answered. The store knew more than it returned: PostgreSQL governed from January 2025 until April 2026, when MongoDB superseded it under a newer architecture decision. Somewhere between storage and the agent, that relationship disappeared. The API returned the records and threw away the relationship between them. That is a very different kind of memory failure, and it is the one this piece is about. The Storage Problem Is Mostly Solved Before going further, it is worth being honest about what is actually new here, because part of this problem was solved before agents existed. Separating when a fact was true from when the system learned it is bitemporal modeling, standardized in SQL:2011 as application-time and system-versioned tables. Edward raised this in the thread, and he is right that the database world has handled "this was true then, this is true now" for over a decade. A well-built store can close a fact's validity window instead of overwriting it, and the past stays explicable. So the interesting problem is not storage. If your store still deletes on update, fix that first, and the literature is waiting for you. The problem this piece is about starts one layer up: even when the store preserves all of it, the retrieval interface usually hands the agent a flat ranked list and throws the structure away. The store solved the problem. The API un-solves it on the way out. A Ranked List Has Nowhere to Put an Edge That phrase is Edward's, from the thread, and it may be the sentence that breaks the whole abstraction. Once you sit with it, the rest follows. Most AI memory interfaces inherited a familiar retrieval shape: give the system a query, get back a ranked list of relevant things. There may be metadata attached, a timestamp, a document id, a source, a confidence value. The fundamental abstraction stays the same. Memory is a bag of items, and retrieval returns the best-matching items. That works well when the problem is finding things. Agentic systems increasingly need memory to do something harder: represent what the system currently knows, what it previously knew, where that knowledge came from, whether it still governs, and how apparently contradictory records relate. A ranked list is a poor representation of that world, because the relationships between records are part of the knowledge, and a list has nowhere to put them. Consider two records: customer refunds require manager approval, and customer refunds under $100 do not. Maybe the second is a correction, because the first was entered wrong. Maybe it superseded the first, because policy changed. Maybe both are true in different jurisdictions and the first simply no longer governs this transaction. Those are not variations of one operation. They make different claims about history. At the storage layer, all three can look like an update. At the audit and retrieval layers, they are fundamentally different events. "No Longer True" Is Not "Never True," and Neither Is "No Longer Governs" CRUD trained us to think in one verb, UPDATE , but durable memory needs at least three, and the third is the one that gets missed. Supersession says the world changed. Policy A was true, Policy B is true now, and A is not wrong, it is closed. Correction says our record was wrong, including during the window an agent may have relied on it, so A was never true. Invalidation is the one worth slowing down for, because it is not a truth claim at all. It is an authority claim. A record can be perfectly true and no longer govern. That distinction is the load-bearing one. A store that collapses these into a single value change can still answer "what is true now" cleanly, and will quietly fail the moment anyone asks "why did the agent approve that transaction on March 17." The answer to that question may depend on a record that is closed, or corrected, or stripped of authority, and that store no longer knows which. Availability Is Not Usage, Even for a Schema Here is the part that should make anyone building this check their own system before writing another feature. Giulio D'Erme read the original thread, then went and counted his own corpus: zero of 152 memos in his memory store, and zero of 59 documents in his docs, declared a validity window or a supersession edge. The engine could read those keys. Nothing that wrote memories ever wrote them. As he put it, availability is not usage, and it applies to schema as much as to tools. This is the failure mode hiding behind every rich schema. You can ship the read path, document the fields, and watch a live API serve a dead feature, because the thing that writes memories, a prompt or a template or another agent, was never taught the keys. A supersession column that nothing populates is not preservation. It is a column. Tae Kim described the same shape from production trade data: the same company surfacing as different nodes depending on whether you asked before or after an acquisition, with the store silently picking one. Stamping the connections with time ranges and returning both versions helped. The part that bit later was that the agent's choice between them still vanished without a trace, which is the next problem. Relevance Is Not Authority The PostgreSQL example exposes the assumption underneath ranked retrieval. A similarity score answers, roughly, "how relevant is this record to the query." It does not answer "which record currently governs." Those correlate, but they are not the same. PostgreSQL might score 0.94 because it contains the exact terminology in the query, while MongoDB scores 0.91 because the migration decision is phrased differently. Retrieval did its job. The agent still gets the wrong answer, because 0.94 > 0.91 quietly became conflict resolution, and semantic similarity never established anything about authority. This is why I have come to think of Memory as Infrastructure rather than memory as a database feature. Once memory participates in consequential decisions, retrieval quality is only one property of the subsystem. Provenance, authority, lifecycle, temporal validity, and correction semantics matter too. The closest memory is not necessarily the memory that governs. Contradiction Is Information Memory systems often treat conflicting records as a retrieval-quality problem: delete the older one, rank the newer one higher, filter one out with metadata. Sometimes that is right. Sometimes the contradiction is the most important thing memory knows. Consider a record from Procurement saying Supplier X is approved for regulated workloads, and one from Security saying Supplier X is prohibited. Both may be inside their validity windows. No supersession may exist. The correct response is not to silently decide which wins. It is to report that the records conflict, where each came from, which authority issued each, and that resolution is required. If the store knows the conflict exists but the API returns two ordinary ranked hits, the disagreement disappears at exactly the moment it mattered most. The Response Type Is Part of the Architecture This is why the fix is harder than adding a metadata column. If memory contains relationships, the response type has to be able to carry relationships. A richer interface might conceptually return something like: { "records": [ {"id": "A", "content": "Production uses PostgreSQL."}, {"id": "B", "content": "Production uses MongoDB."} ], "relationships": [ { "type": "supersession", "from": "A", "to": "B", "effective_at": "2026-04-15T00:00:00Z" } ] } The precise schema is not the point, and I am not proposing that JSON as a standard. The conceptual change is that the response is no longer a list of memories. It is a representation of a knowledge state, one that can carry contradiction, supersession, correction, invalidation, provenance, and authority as first-class content. Once those relationships affect agent behavior, they cannot stay trapped in the storage layer. Two honest problems come with that, and both surfaced in the thread and then got worse the more Edward and I pushed on them. The first is budget, and it turns out to be deeper than allocation. A ranked list is impoverished, but it is cheap, and top_k is a clean way to decide what to drop. The moment a response carries facts, relationships, authority, provenance, and prior decisions together, the problem stops being ranking and becomes allocating a finite context budget across different kinds of knowledge. A lower-ranked authority edge may matter more than the next highly relevant fact, and dropping a supersession relationship can change the meaning of the records that survive. The tempting fix is to select the edges after ranking, as a post-filter on whatever top_k returned. Edward's counter is the part that reshaped my thinking: to know whether a supersession edge is worth carrying, you already have to be holding the record it supersedes. Edge hydration therefore cannot be a post-filter. It has to influence which candidates are considered in the first place, which mea

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.