Zero Data Retention Won't Stop Your Own Logs From Leaking Prompts

Here is a prompt that a lot of production apps send to an LLM today:

Analyze the attached six months of transactions for John Smith, account 123456789. Identify unusual spending patterns.

Say your provider offers zero data retention (ZDR) for the endpoint you call. Great. Now check where else that string lives: your request logs, your tracing tool, your retrieval cache, the tool-call payloads of your agent. ZDR covers none of those.

We build this at Protecto, so we'll flag our bias up front: we think the fix is to send less, not to trust a retention setting more. Below is the reasoning, plus a toy pattern you can run.

The model may be stateless. Your system isn't.

"We don't train on your data" and "we don't retain your data" are different promises. A provider can skip training and still store content for conversation history or safety monitoring. Retention windows measured in days exist at some providers, so check each provider's current documentation and your contract.

So there are three separate controls, and each answers one question:

None of them replaces the others. Also check each commitment against the specific service, endpoint, and features you use, because eligibility often differs by model and feature.

What ZDR does not cover

• Disclosure. The external model still receives whatever you send.
• Geography. Storage and inference are separate questions.
• Your own copies. Application logs, observability platforms, retrieval pipelines, and connected tools keep data independently of the provider.
• Exceptions. Ask directly what abuse monitoring excludes, what exceptions remain, and whether specific features store content.

The better question is not "how long does the provider keep it?" It is "where does sensitive information travel, and where does each copy persist?"

Ask why you sent it before asking how long it stays

Most tasks do not need the real identifiers. This is the same request with values swapped for placeholders:

Analyze the attached six months of transactions for CUSTOMER_001, account ACCOUNT_001. Identify unusual spending patterns.

The analysis still works. Your app keeps the mapping and restores real values only for authorized recipients when it shows results.

Here is a minimal sketch of the idea. It is a toy: regex-based, in-memory, English-only, and it will miss things. It exists to show the shape of the flow, not to protect production data.

Two trade-offs worth testing before you rely on this:

• Leftover signal. Merchant names, locations, dates, and unusual transactions can still reveal who someone is.
• Over-removal. Strip too much context and answer quality drops.

Run the protected workflow against representative tasks, not one happy-path prompt.

Two architectures, different disclosure

Direct disclosure

Raw prompts go from your app to the external LLM. The provider receives every sensitive value, and your logs probably do too.

Protection before disclosure

A layer masks or tokenizes sensitive values before the model call, and restores them only through a policy-governed step. If the layer runs inside your environment, original values and token mappings stay inside your boundary.

Agents make this harder. A single masking step at the start does not cover a workflow where a later tool call queries a database and hands raw records back to the model. Protection has to apply at every step: retrieval, tool calls, responses, and logs.

For transparency on what is what: the snippet above is a throwaway example written for this post. Protecto Vault is our commercial product and is not open source. It tokenizes values before the model call and restores them under policy, and it can run in your environment. It reduces what is disclosed, while ZDR and residency still matter for whatever you do send.

Questions to ask

Takeaway

No-training, ZDR, and residency answer different questions. Data minimization asks a prior one: did the value need to leave at all? Pick one workflow and start there. A provider cannot retain what you never sent.

We're curious how others handle this. Where do raw prompts end up in your stack that you didn't expect: logs, traces, or tool payloads? Corrections welcome, especially on provider retention details, which change often. We'll fix the post if we got something wrong.