We Audited Our Claude Code Setup Against Anthropic's Own Context-Engineering Rules - Here's What We Found

What the blog post actually says

Stripped of marketing language, the post boils down to five concrete rules:

• Keep CLAUDE.md lightweight. Describe gotchas and non-obvious patterns, not everything you know about the repo. Organize by relevance, not comprehensiveness.
• Progressive disclosure. Load context at the right time - skills, references, and detail should be pulled in when needed, not front-loaded into every session regardless of task.
• Trust the model's judgment. Remove redundant guardrails and standing instructions that the newer models don't need spelled out every time.
• Rely on automatic memory, not manual dumps. Don't hand-maintain a giant preferences block in a markdown file - let the memory system surface the right thing at the right time.
• Design tools and interfaces, not prose. Push instructions into tool schemas and parameter design rather than repeating them in the system prompt.

None of this is radical. It's the same discipline good engineers apply to code: don't load everything into scope just because you might need it, don't repeat yourself, don't keep dead configuration around because removing it feels risky. The interesting part is applying it literally, to an actual production Claude Code setup, and seeing what falls out.

What we found when we actually measured it

We started by tracing exactly what gets injected into the context window at SessionStart, since that's the one moment every single session pays the cost. The .claude/settings.json SessionStart hook was running a chain of gps commands. Three of them, in sequence, all touched the exact same data:

• gps preferences --write # bakes a markdown block into CLAUDE.md
• gps preferences --markdown # prints the same list to stdout
• gps prime # emits a session primer that overlaps both

--write persisted a <!-- gps:auto-prefs -->...<!-- /gps:auto-prefs --> block directly into CLAUDE.md, which is read every session. --markdown printed the identical list again as hook stdout, which Claude Code surfaces as a second system reminder. prime added a third pass that substantially overlapped both. Same ~60-line list, injected three times, every single session, before any actual task began. That's rule #1 and #4 violated in one shot: a hand-persisted dump in CLAUDE.md (the opposite of "lightweight, gotchas only") that was also duplicated by the very memory system that was supposed to replace it.

Then we looked at what was actually in that preferences list, because 60 lines of genuine cross-cutting invariants would at least be defensible even if wastefully injected three times. It wasn't. Pulling the raw store (.gps/preferences.yml) showed 105 entries, and a good two-thirds of them weren't standing preferences at all - they were things like:

• A verbatim quote from an eval judge's commentary: "Judge weakness: 'never specifies idempotency for event consumers.'"
• A code comment lifted out of context: // replace, never append
• A fragment of a test assertion: test('refresh() replaces list, never appends', () async { ...
• Duplicate phrasings of the same secrets rule, recorded three separate times with three separate IDs.

The mechanism responsible was gps capture-preference --emit and gps capture-directive --emit, both wired into the UserPromptSubmit hook - meaning they fired on every single message in every session. Their logic was a bare regex cue: anything matching \b(?:please\s+)?never\b or \balways\b, recorded as a standing, always-injected "preference," with source: auto and no expiry. It didn't matter whether the match came from the user's actual instruction, a pasted log, a quoted test file, or the agent's own prior output describing what an eval judge said. If the substring was there, it got written to disk and re-injected every session forever.

This is the sharpest version of the trap the blog post warns about: "automatic memory" sounds like the responsible, judgment-trusting choice - right up until the automation itself has no judgment. A regex doesn't know the difference between a user's hard rule and a stray quote in a transcript. Left alone, it just accumulates noise at the same rate it accumulates signal, and every session pays for both.

On top of that, we found two plugins - caveman (terse response style) and ponytail (anti-overengineering behavioral rules) - set to always-on in the global enabledPlugins config. Both inject their full rule text (roughly 40 lines each) at every SessionStart, regardless of whether the task at hand has anything to do with response verbosity or engineering discipline. Useful personas, wrong default: standing injection instead of on-demand invocation is exactly the "load everything just in case" pattern rule #2 argues against.

The fix

We worked through it in four concrete steps.

1. Collapse the triplicated emission. Dropped --write and --markdown from the SessionStart hook, keeping a single gps prime pass. One source of truth, one injection per session.

2. Strip the manual dump from CLAUDE.md. Removed the <!-- gps:auto-prefs --> block entirely. CLAUDE.md went back to being what rule #1 asks for: hand-curated gotchas, not an auto-regenerated preference log.

3. Prune the store. We pulled the full 105-entry list and manually classified every one. About 35 were genuine, cross-cutting invariants worth injecting into every session regardless of task - secrets handling, PII rules, git-push restrictions, migration reversibility, PCI scope boundaries. The other 70 were dropped: judge commentary, code fragments, near-duplicate phrasings, symbol-specific one-off fixes that only matter when that exact piece of code is touched again (those still exist - just surfaced on demand via gps prepare_edit when relevant, not blasted into every session).

4. Fix the actual leak, not just the symptom. This is the part worth dwelling on, because the first three steps only clean up what had already accumulated - they don't stop it from happening again. The regex-based capture-preference / capture-directive hooks were still wired in and would refill the store with the same class of noise within days. We replaced them with a small filtered wrapper script that:

   • reads only the literal current user prompt ($CLAUDE_USER_PROMPT), never the transcript, never the agent's own output, never quoted or pasted content;
   • requires an actual imperative sentence shape - the text has to start with never, always, don't, must, or should, not just contain the word anywhere;
   • rejects lines that look like code, quotes, or table rows (//, #, |, `, >);
   • checks the existing store for a near-duplicate before writing anything new.

We disabled caveman and ponytail as always-on plugins and left them invokable via their slash commands, on demand, exactly when their behavior is actually wanted.

The mistake worth admitting

Midway through pruning that 105-entry list by hand, we cut a handful of rules that were real, not noise - a "reference the config key, never the value" secrets-handling convention, a backend mock-fallback contract invariant, an API schema-evolution rule ("add fields, never remove"), a known config-server caching gotcha. They looked narrow enough to fold into "one-off" at a glance, but they were genuinely cross-repo conventions, not symbol-specific trivia. The user caught it with a simple question: "is it losing existing functionality?" That's the right question to ask after any aggressive prune, automated or manual - and it's the reason we're writing this section at all. Nothing was destroyed; the full pre-prune store was still sitting in git history, recoverable with one git show. But it's a useful reminder that "delete the noise" and "delete everything that isn't obviously load-bearing" are not the same operation, and doing the second by accident while intending the first is an easy mistake to make when you're skimming 105 entries under time pressure.

Verifying the automated version wasn't just automating the same mistake

Before trusting the new filtered capture script, we tested it against an isolated scratch copy of the .gps store - not the live repository - feeding it a mix of a genuine directive and the exact kind of noise that had polluted the original store:

Never log raw request bodies for the payment endpoint.
Always validate the webhook signature before processing.
The judge said: never sent to the network, that's just eval commentary.

The result: both real directives were captured and written to the store correctly. The judge-commentary sentence was rejected, because it doesn't open with an imperative verb - it opens with "The judge said." That's a small test, but it's exactly the discriminator the original regex lacked, and it's cheap enough to run before wiring anything back into a live hook.

What this actually bought us

Token-wise, the honest estimate is roughly ~1,200+ tokens injected at every SessionStart before the fix, down to ~400 after - a triplicated 60-line list plus a noisy 105-entry store plus two always-on plugins, versus one clean primer pass plus a curated 35-entry store plus plugins that only cost anything when explicitly invoked. That's not a huge absolute number in isolation, but it's paid on every single session, compounds across a team, and - more importantly than the raw token count - it was diluting the signal-to-noise ratio of the one list that's supposed to carry the invariants Claude should never violate. A 35-item list of real rules is something a model can actually hold onto. A 105-item list where two-thirds is eval-transcript noise trains exactly the wrong instinct: skim past it, because it's mostly not going to matter.

The generalizable lesson

None of the individual fixes here are exotic - collapse duplicate emissions, prune a noisy store, tighten a regex, gate a plugin behind explicit invocation. The useful part is the audit discipline itself: pick a set of concrete, checkable rules (Anthropic gave us five good ones), and then actually trace what your own automation does against them, hook by hook, rather than assuming that "we have automatic memory" or "we use skills" means the rules are satisfied by construction.

Automation that was designed well can still drift into violating the very principles it was built to serve - usually quietly, one auto-captured preference at a time, until someone asks what's actually in the store. If you're running Claude Code with any nontrivial hook setup - gps, a custom memory layer, auto-capture of any kind - it's worth doing the same trace. Open the SessionStart hook chain, count how many times the same information gets emitted, and go look at what your "automatic" capture mechanism has actually been recording for the last few months. You may be surprised what's in there.