Your AI Agent Stack Is Solving The Wrong Problem

Your AI Agent Stack Is Solving The Wrong Problem The setup everyone is sharing Which MCP servers to install. Which skills to keep in your repo. Which agent framework to use. How to write your AGENTS.md . How to split one agent into researcher, planner, coder, and reviewer. How to wire Slack, GitHub, Notion, Postgres, Stripe, your calendar, and your file system into one increasingly capable loop. Some of that advice is useful. It is also aimed at the wrong layer. What becomes real after the agent uses a tool matters more than whether it can reach the tool. Can it read the customer record, or change it? Can it draft the refund, or issue it? Can it open a pull request, or merge it? Can it propose the vendor response, or send it under the company name? Once an agent can act through tools, the real system is no longer the model. The real system is the contract stack around the model. That is the part most setup guides skip. Access is reach. Agency is permissioned action. Imagine the demo. The agent can read Slack. It can search email. It can query the CRM. It can open GitHub issues, check billing records, browse docs, edit a spreadsheet, draft a customer reply, and call three internal APIs. Everyone in the room calls it powerful. That is the first mistake. The agent has reach. It does not yet have governed agency. Access tells you what the agent can touch. Agency tells you what the agent is authorised to decide, under which conditions, with what proof, and with what consequence after failure. That distinction sounds small until the first bad run. A read-only research assistant can waste time. An agent with billing access can create obligations. An agent with email access can speak for the company. An agent with deployment access can turn a wrong inference into infrastructure. More tools do not automatically make the agent more agentic. More tools expand the surface on which judgement has to be engineered. The tool stack is visible. The contract stack is load-bearing. The visible agent stack is easy to list: model, prompt, memory, tools, MCP servers, subagents, framework, evals. That stack matters. It is also not the operating system. The operating system is the set of contracts each layer creates. What is the agent for? What state may it see? What state may it preserve? Which tools may it call? Which tools are intentionally absent? What can it change? What must it prove before the change becomes binding? What does the harness log? What does the evaluation score actually cover? When does the agent ask, abstain, or escalate? What permission disappears after a bad run? That is the real setup. Not the list of tools. The set of boundaries that decides what the tools mean. The generic setup stack asks what you connected. The contract stack asks what you can trust. An agent is a control loop, not a prompt with ambition An agent is an outer control loop wrapped around a generator. It plans, reads state, chooses tools, acts, observes, repairs, escalates, and decides whether to continue. The failure rarely sits in one glamorous place. It can sit in the planner. It can sit in retrieval. It can sit in a tool description. It can sit in retry logic. It can sit in a hidden assumption about whether the world waits while the agent thinks. That is why framework comparisons are often less useful than they look. The distinction that matters is which parts of the loop are explicit enough to inspect. If planning is hidden inside one long natural-language instruction, you cannot repair planning without rewriting the whole prompt. If memory is just a growing transcript, you cannot tell whether the agent remembered, retrieved, inferred, or hallucinated. If tool choice is unlogged, you cannot tell whether the answer is wrong because the model reasoned badly or because it called the wrong thing. If evaluation is one final pass/fail number, you cannot tell whether the agent failed at discovery, parameters, sequencing, recovery, escalation, or judgement. Agents do not become reliable when the setup becomes more impressive. They become reliable when failure has somewhere specific to land. MCP is not magic glue MCP matters. Skills matter. Connectors matter. But their importance is often described backwards. The lazy version says MCP is valuable because it gives agents more tools. The better version says MCP is valuable because it makes the tool boundary explicit enough to inspect, version, test, authorise, and debug. A tool is not neutral plumbing. A tool description tells a nondeterministic system what an action means. The name, parameters, return shape, error messages, and allowed mutations all change behaviour. A tool built for a human developer is not automatically a good tool for an agent. Humans carry missing context. Agents need the contract written down. That is why more tools can make an agent worse. At small scale, tool access feels like freedom. At larger scale, tool access becomes search. The agent has to identify the right tool, pass valid parameters, recover from partial failure, and avoid inventing a successful trace when the tool call failed. If you expose every API endpoint as a tool, you do not have a powerful agent surface. You have a vocabulary problem with write access. The mature move is not “connect everything.” The mature move is to design the smallest tool surface that lets the agent do the job, then make every tool contract legible. What does the tool do. When should it be used. What the return value proves, and what it does not prove. What failures look like. Which calls are read-only, which mutate state, which require approval. Where the trace goes. That is how you stop a transcript from becoming the only place your operating system exists. Skills are not prompt snippets The same mistake happens with skills. People treat skills as better prompts: a SKILL.md , a few examples, some instructions, maybe a script. Useful. Portable. Easy to share. But a serious skill is not a prompt snippet. It is packaged operating knowledge. It should contain a trigger, a procedure, a boundary, gotchas, and a failure mode. The “gotchas” are usually the most valuable part. The model often already knows the happy path. What it does not know is your local scar tissue: which API lies, which file must not be edited, which naming convention breaks deployment, which customer segment changes the policy. That is why generic skill catalogues have a ceiling. They can teach a model the common workflow. They cannot teach it which parts of your workflow are load-bearing unless you package that knowledge yourself. Skills are valuable because they let operational knowledge travel across sessions and agents. They are dangerous when they activate at the wrong time, compose implicitly into deeper graphs nobody intended, or grant state-changing behaviour without a permission contract. The real question is sharper: When this skill activates, what decision is it allowed to influence? If nobody can answer that, the skill is just a more durable way to make the wrong move. Memory is governed state, not a bigger past Memory has the same problem. Every agent product wants to promise memory. It sounds obvious. The agent should remember the user, the project, the codebase, the customer history, the prior decision, the mistake from last time. But memory is not “more context.” Memory is a four-part contract: what gets written, how it is organised, how it is retrieved, how it is governed. If the agent writes too much, memory becomes sludge. If it summarises badly, memory becomes distortion. If it retrieves by similarity alone, memory becomes vibes with citations. If it never forgets, memory becomes context poisoning. If it cannot show why a memory was used, memory becomes an invisible authority. The memory question worth asking: Which state should survive because it will improve future decisions, and which state should expire because it will poison them? That is a contract question. It is also why a 500-word, well-maintained project note can outperform a giant chat history. The smaller note has a job. The transcript merely has volume. The harness is where autonomy becomes measurableMost agent demos make the model look like the protagonist. In production, the harness is the protagonist. Top comments (0)