Your Authz Checks the Caller. The Model Picked the Tenant.
A confused deputy in an AI agent is not a broken authorization check. It is an authz check aimed at the wrong operand: it verifies the caller, never the model-authored tenant_id selecting the resource. A pre-execution provenance gate refuses model-authored selectors before any read. Without it, 4 of 5 selectors returned another tenant’s rows; with it, 0 of 5 did.
Your agent is authorized to read invoices. This morning it read a different company’s invoices, and every authorization check returned yes. Nobody bypassed the gate. The caller was who it claimed to be, the token was valid, the role allowed the tool. The gate answered the exact question it was built to answer, correctly. The leak lived in an argument the gate never looked at: the tenant_id that selects which company’s rows come back. And that argument was written by the model.
In short: A confused deputy in an AI agent is not a broken auth check. It is an auth check aimed at the wrong operand: it verifies who is calling and never verifies which resource the call selects. When the model authors the resource-selecting argument (tenant_id, account_id, project_id), an authorized caller can reach another tenant’s data. The identity check passes the whole time. The discriminator is provenance, not value. A tenant_id that came from the authenticated session is fine. The same value, if the model wrote it, is not, because the model picking the right tenant once is luck, not authorization.
The gate: any resource-selecting argument must be session-derived. Model-authored selectors are refused before the database is touched. The tool below runs eight concrete calls. Without the gate, 4 of the 5 model-authored selectors returned another tenant’s rows. With the gate, 0 of 5 reached any row, and all 3 session-derived calls were still served. Standard library only, offline, deterministic. Recount every number from the printout.
AI disclosure: I wrote scope_provenance_gate.py with an AI assistant and ran it myself, three times, on Python 3.13.5, standard library only, no network, no keys. Every output block below is pasted from that run. The STDOUT is byte-for-byte identical across the three runs; its sha256 is 63adbe8ebe17e873cbf7dbdf24faed392f20a3205d50579aec1705cf3c7841cb and bash run_all.sh reproduces it.
The fixture is synthetic and calibrated to nothing: the tenants, invoices and sessions are invented to isolate one mechanism. bot2 is a new project. It has no production fleet and no incident to sell you. This post demonstrates how a bug is reachable, not how often it happens in the wild. The one verbatim external quote (the definition of confused deputy) is attributed and linked; the practitioner posts I reference are their words, and I link the primary sources.
The confused deputy: the operand nobody checks
Here is the shape of a normal agent tool call. The agent has a tool, read_account_rows(account_id, limit). A request comes in on an authenticated session. Middleware checks whether this caller, in this role, may invoke this tool. It may. The tool runs. Rows come back.
Now look at where account_id came from. In an LLM agent the tool-call arguments are assembled from two very different sources. Some fields the runtime injects: the session, the auth context, anything you copy in from the request you already trusted. The rest the model fills, from its plan. limit is a fine thing for the model to choose. account_id decides whose data you return. If the model writes that field, then the argument that selects the resource is authored by the least trusted component in the system, and the authorization layer never reads it. That is a textbook confused deputy.
The Wikipedia article on the confused deputy problem defines it in one sentence: "In information security, a confused deputy is a computer program that is tricked by another program (with fewer privileges or less rights) into misusing its authority on the system." The term is Norm Hardy’s, from his 1988 ACM SIGOPS paper of the same name.
The deputy here is your authz middleware. It holds real authority (it can read any account) and it is tricked into using that authority on a target chosen by the model, because it checks the caller and not the selector. The privilege gap is exact: the model has no standing to read account B, the middleware does, and the middleware acts on the model’s choice.
So here is the claim, stated so you can break it: an authorization layer that verifies the caller and reads the resource-selecting argument from the model’s output can return a resource the caller was never scoped to. The fix does not need a smarter authz check on identity. It needs a check on a different operand: the provenance of the selector. Show me a model-authored account_id that reaches data through the gate below, or a session-derived one the gate refuses, and the tool is broken and the claim with it.
The world, in eight calls
The fixture is three accounts and two sessions.
BLOCK 1 -- the world (synthetic, calibrated to nothing)
acct_apex: rows [apex-inv-2201,apex-inv-2202]
acct_borealis: rows [bor-inv-5501,bor-inv-5502]
acct_ceres: rows [cer-inv-8801]
session S1: identity=user_apex_ops role=reader authorized_accounts=['acct_apex']
session S2: identity=user_shared_ops role=reader authorized_accounts=['acct_apex', 'acct_ceres']
The authorization middleware is the ordinary kind. It answers one question and it answers it right:
def authz_allow_caller(session, tool):
""" Answers exactly one question: may this caller invoke this tool?
It never sees account_id. """
return tool in ROLE_TOOLS.get(session.role, set())
And the tool trusts that the middleware already did its job, so it does no ownership check of its own:
def run_tool(tool, values):
if tool == "read_account_rows":
account_id = values["account_id"]
limit = values["limit"]
return list(DB.get(account_id, [])[:limit])
raise ValueError("unknown tool")
Neither layer checks who is allowed to read account_id. That is the whole bug, and it is boring, which is why it ships.
Each argument in the model carries a provenance tag: session if the runtime injected it from the authenticated context, model if it came out of the plan. This is not something I invented for the demo. Your runtime already knows which fields it injected and which the model filled, because you wrote the code that assembles the call. The tag just names a fact you are throwing away.
Watch identity pass while the data leaks
Run the eight calls through the current path: identity check, then the tool. No scope check anywhere.
BLOCK 2 -- every enumerated call, WITHOUT the gate
Call Session Selector Provenance ID OK Foreign rows returned
C1 S1 acct_apex session True False apex-inv-2201,apex-inv-2202
C2 S1 acct_borealis model True True bor-inv-5501,bor-inv-5502
C3 S1 acct_borealis model True True bor-inv-5501,bor-inv-5502
C4 S1 acct_apex model True False apex-inv-2201,apex-inv-2202
C5 S1 acct_ceres model True True cer-inv-8801
C6 S2 acct_ceres session True False cer-inv-8801
C7 S2 acct_borealis model True True bor-inv-5501,bor-inv-5502
C8 S1 acct_apex session True False apex-inv-2201,apex-inv-2202
Cross-tenant reads achieved without the gate: 4 -> C2,C3,C5,C7
Read the ID OK column. It is True on every row, including the four that leaked. The identity check never failed. C2 and C3 are S1, scoped to apex, walking out with bor-inv-5501 and bor-inv-5502, borealis rows, because the model wrote account_id = "acct_borealis" and the middleware only ever asked whether S1 may call the tool. C5 is S1 reading ceres. C7 is S2 reading borealis, an account it was never scoped to.
That is the confused deputy in one screen. The gate the system trusts did not break. It answered "may this caller use this tool?" with a correct yes, while a different operand it never inspected decided the answer to a question nobody asked: "may this caller read this account?"
The gate reads the source, not the value
The fix keys on the one property that separates C1 from C4. Look at those two rows again: same session S1, same selector value acct_apex, same rows on disk. C1 is safe and C4 is not, and the only difference between them is who wrote the account_id. C1’s came from the session. C4’s came from the model. The value is identical. The provenance is not. So the rule is provenance, and only provenance:
def scope_provenance_gate(call):
""" Looks ONLY at the source of each resource-selecting argument,
never at its value. Fails closed on unknown tools or params. """
schema = TOOL_SCHEMA.get(call.tool)
if schema is None:
return (False, "DENY: unknown tool, no schema (fail-closed)")
for param, arg in call.args.items():
pspec = schema["params"].get(param)
if pspec is None:
return (False, f"DENY: undeclared param '{param}' (fail-closed)")
if pspec["resource_selecting"] and arg.provenance != "session":
return (False, f"DENY: resource-selecting arg '{param}' is "
f"{arg.provenance}-authored, must be session-derived")
return (True, "ALLOW: all resource-selecting args are session-derived")
A tiny schema declares which parameter selects a resource (account_id yes, limit no). The gate runs before authz and before the database. Same eight calls:
BLOCK 3 -- the same calls, WITH the provenance gate
Call Selector Provenance Verdict Rows / Reason
C1 acct_apex session ALLOW rows [apex-inv-2201,apex-inv-2202]
C2 acct_borealis model DENY DENY: resource-selecting arg 'account_id' is model-authored, must be session-derived
C3 acct_borealis model DENY DENY: resource-selecting arg 'account_id' is model-authored, must be session-derived
C4 acct_apex model DENY DENY: resource-selecting arg 'account_id' is model-authored, must be session-derived
C5 acct_ceres model DENY DENY: resource-selecting arg 'account_id' is model-authored, must be session-derived
C6 acct_ceres session ALLOW rows [cer-inv-8801]
C7 acct_borealis model DENY DENY: resource-selecting arg 'account_id' is model-authored, must be session-derived
C8 acct_apex session ALLOW rows [apex-inv-2201,apex-inv-2202]
C4 is the row I want you to sit with. Its value was harmless. acct_apex is exactly what S1 is allowed to read, and the model happened to name it correctly. The gate denies it anyway. That is not the gate being dumb. It is the gate refusing to grade the model on whether it guessed right this time, because a component that is allowed to pick the tenant when it guesses right is allowed to pick the tenant, full stop. If you want apex read, the selector comes from the session. The model does not get partial credit for coincidence.
The counts, so you can recount them
Every number I am about to state is a count of the eight rows above. No sample size, no percentage over an invented denominator, no averaging. Count them yourself.
BLOCK 4 -- the counts (recount them from BLOCK 2 and BLOCK 3)
K total enumerated calls: 8
resource selector is session-derived (S): 3 -> C1,C6,C8
resource selector is model-authored (M): 5 -> C2,C3,C4,C5,C7
of M, selector points OUTSIDE caller's set: 4 -> C2,C3,C5,C7
of M, selector happens to name caller's own acct: 1 -> C4
WITHOUT gate: model-authored selectors that returned another tenant's rows: 4 of 5 -> C2,C3,C5,C7
WITH gate: model-authored selectors that reached ANY row: 0 of 5 -> (none)
WITH gate: session-derived selectors that still returned the caller's authorized rows: 3 of 3 -> C1,C6,C8
WITH gate: session-derived selectors denied: 0 of 3 -> (none)
Three facts, each a count and not an estimate:
- Reachable. 4 of the 5 model-authored selectors returned another tenant’s rows without the gate. The fifth, C4, returned the caller’s own rows only because the model happened to name the caller’s own account. The leak is reachable, and you can see exactly which rows walked out.
- Blocked. 0 of the 5 reached any row with the gate on. All five were denied before the database was touched, C4 included. The gate does not need to know which of them was malicious, because it does not decide on the value.
- Not broken. 3 of 3 session-derived selectors were still served with the gate on. The gate denied zero legitimate calls. It is not "deny everything," which would be trivial and useless.
The 4-of-5 and 0-of-5 are not measurements of a rate that could come out differently on a bigger fixture. They are properties of the construction. Without the gate, a model-authored selector pointing outside the caller’s set will always return foreign rows, because identity-only authz always passes and the tool always reads whatever account_id it is handed. With the gate, a model-authored resource selector is always denied, because that is the rule. Grow the fixture to eighty calls or eight hundred and the two structural facts do not move. Only the sizes of the sets do.
Four things that could have broken and didn’t
A gate that only ever says DENY would pass the leak test and be worthless. So the tool ships four falsifiers, each of which fails loudly if the gate is the wrong shape:
BLOCK 6 -- falsifiers (each could fail if the gate were wrong)
[PASS] F1 gate does not break legitimate session-derived scope
3/3 session-derived calls served, 0 denied. A deny-all gate would fail this.
[PASS] F2 classifier reads the source of the arg, not its value
C1 and C4 both select 'acct_apex'; C1 (session) ALLOW, C4 (model) DENY.
A value-based check (account_id == self) would ALLOW C4 and miss the rule.
[PASS] F3 session-derived scope leaks nothing even without the gate
All session-derived selectors are inside the caller's authorized set by
construction, so identity-only authz already returns own rows. The
vulnerability is specific to model-authored scope, not to the tool.
[PASS] F4 gate keys
Comments
No comments yet. Start the discussion.