I Found Three Broken Rules in My Own AGENTS.md
For about a year I've done all my development through workflows, and most of that year has gone into maintaining them. When a run came out wrong, I read the run, found the part of the mechanism that produced it, and changed that. Not the prompt for next time. The mechanism. That habit was cheap insurance when it was optional. It is not optional now.
What changed
Older models ignored a lot of what you told them, and that was load-bearing. If two of your skills gave conflicting instructions, the model quietly picked one. If a rule said "always produce X," you got X about half the time, which was usually the right amount. Your setup could be full of contradictions and you'd never find out.
Better models follow instructions. They follow the contradictory ones too, and the result shows up in the run: a skill fires when it has nothing to contribute, a mandatory step blocks work that didn't need it, a rule demands an artifact nobody reads. Instructions that used to sit there doing nothing now actually affect what happens.
Three things that decide how a session goes
- What you asked for. Your messages are the prompt. If "done" isn't defined, the agent invents a definition, and it won't be yours. Contradictions are worse than ambiguity, because a capable model acts on both sides of one.
- How the work was run. AI development is management: delegation, process, and technical decisions. That was true when one agent did the work, and orchestration made it structural. You manage a main agent, the main agent manages its subagents, and you're two levels away from where the work happens. Once that's the shape, you need to know who owns each decision and what counts as finished, or work gets lost between the levels. Most setups have none of that written down anywhere the agent can see.
- What was instructing the agent. Not your skills as files. Your skills as they actually combined in that session, with your AGENTS.md and whatever else was live at the same moment. A rule that reads fine on its own can still be wrong next to a skill that fired at the same time.
Keeping skills current is the part nobody does
Plenty of people have a directory full of skills. Very few can tell you which ones fired in their last session, which ones changed a decision, and which ones loaded a few thousand tokens and contributed nothing.
I don't think that's negligence. Creating a skill has an obvious method and an immediate result. Improving one doesn't. There's no natural moment where you find out that a skill you wrote in March is duplicating a rule you added in July, or that it triggers on work it can't help with. If you've never had a way to measure the effect, you have no reason to look.
What made this tractable for me was that a workflow run leaves artifacts, so when something goes wrong there's a trail.
What a year of that looks like
claude-code-workflows is at 133 releases since October 2025, and the shape of them tracks what the models were doing at the time.
- Early on I was adding mechanism, because the model needed the structure to stay on task. Then the models got better and I spent releases taking it back out.
- In v0.23.0 I removed the current-date retrieval step from six agents.
- In v0.26.0 I deleted the rule-advisor agent and the task-analyzer skill outright, because the agents that needed those rules already loaded them directly, so the selection step added a round trip without changing what ran.
- v0.24.0 removed a whole recipe and cut the planning templates down to what implementation actually consumed.
Lately the changes are about delegation: who owns a decision, what a subagent is told, what has to come back before the parent can act. None of that was foresight. It was using the thing on real work, watching it fail, and going back to find the part that produced the failure.
ai-coding-project-boilerplate went through the same cycle. The trail exists for ordinary sessions too. It's just harder to read.
Session Doctor
Session Doctor reads one of your saved Claude Code or Codex sessions and reports what to change across all three. The three passes run separately, so a hunch from the first doesn't bias the other two, and each finding names where it happened, what it actually changed in the run, and the smallest change likely to prevent a repeat.
I pointed it at a small two-turn session in one of my own repositories. The findings:
- The first finding was a rule in that repository's agent definition file, a
CLAUDE.mdpredating theAGENTS.mdstandard, requiring a task breakdown, a date check, and a rule-advisor call before any work begins, with the words "required for all work, no exceptions." None of them ran, and the task completed correctly. I deleted the date step from my workflow in July. I deleted rule-advisor entirely in September, on the grounds that it added a round trip without changing what ran. The instruction file in that repository never got the message. - The same file names a tool that doesn't exist. It says
mcp__local-rag__query_documents. The real identifier ismcp__mcp-local-rag__query_documents. Anything following that instruction literally fails. - Two sources both claimed to own what happens when a task starts, in incompatible ways. The file's gate said it ran first with no exceptions, while its own routing rule sent the same trigger to a skill that has no gate at all. One of them was always going to be skipped silently. In this session the skill won, and nothing recorded the decision.
Three defects in my own setup, in a two-turn session, in a repository I maintain.
Try it
Claude Code:
/plugin marketplace add shinpr/agent-clinic
/plugin install session-doctor@agent-clinic
Codex:
codex plugin marketplace add shinpr/agent-clinic
codex plugin add session-doctor@agent-clinic
Then run /recipe-diagnose in a repository you've been working in, or $recipe-diagnose in Codex. With no argument it picks up your most recent session there and asks you to confirm before it starts. The README covers the rest.
Run it on the bad one
The session worth diagnosing is the one that went badly, and the instinct is to close the tab and start over with a better prompt. That throws away the best record you have of how the thing actually failed. A capable model working inside a setup that contradicts itself produces work that looks like a model problem. That matters because you can't fix the model, but you can fix the setup around it.
If you run it, I'd like to hear what it found. The defects that show up across a lot of setups are the ones worth writing about next.
Comments
No comments yet. Start the discussion.