Reddit - r/MachineLearning

Is there a name for this: local model holds your context, cloud model never sees the raw data? [D]

The problem

I want frontier-model reasoning over my actual life. Years of notes, contacts, decisions - the whole substrate. Not a stateless chat box.

Every existing option fails one of two ways. Cloud memory features solve capability by absorbing your context onto someone else's servers, and the trust model is a settings page. Local-first solves privacy by refusing capability, so the moment you need real reasoning you're copy-pasting into a cloud tab. Redaction by vibes. Enterprise DLP filters strings under an org policy. There's no "you" in that loop.

The current solution

What I have running - and I mean running for me, not a product - is this: a local model process is the only thing that touches the personal store. Anything bound for a cloud model crosses a gate where the local side decides scope, redacts, and logs the crossing. The cloud model gets treated as very smart untrusted labor. It never pulls. The inner side never pushes raw.

Why trust needs to be structural

Why I stopped trusting the polite version: I had my dad message the assistant and try to social-engineer it. It refused his bait questions fine, then leaked my real phone number in the middle of explaining one of the refusals. Nobody asked for the number. The model just reached into memory and put it in prose. That convinced me the fix has to be structural, not behavioral. A capable model is a great employee and a terrible vault.

Honest state and open questions

Honest state: the boundary works, logging is basic, redaction is rules plus my own review, nothing formally verified, no repo to point at yet. Which is part of why I'm asking.

This feels too obvious to not already exist. What already does this that I've failed to find? And where does it fall over?

  • Inference-time leakage from the redacted slices?
  • The local gatekeeper being too dumb to catch a clever extraction?

Tell me I'm wrong.

Comments

No comments yet. Start the discussion.