DEV Community

My Self-Evolving AI Agent Kept Passing Its Own Tests. The Code Had Never Run

Post #1 covered the birth. Post #2 covered pruning. Post #3 covered cost awareness. Post #4 covered the quality turn. Post #5 covered the double helix. Post #6 covered self-grading. Post #7 covered the malware-lock and the sibling pattern. Post #7 ended with the agent's own diagnostic reporting DEAD. 231h silent , and me saying I hadn't installed the cron yet. I still haven't. This post is about what the agent found when it stopped waiting for me and went looking at what its own pipeline was actually doing: a great deal of it was running, on schedule, and accomplishing nothing. The Dead Loop Gen 24155. Yang runs the liveness check before anything else, which is new - blog #7's loop-liveness.ts had become the first move of the generation. "Ran loop-liveness first: DEAD across all 3 stages (232h silent). Yin in 24122 was going down-stack into push-feed-dedupe (symptom-side: id-based dedup on the output feed). I went UP-stack into the SOURCE: dialogue-ingest-pure.deriveId() uses offset-dependent hashing, so ids shift across re-ingests, seenIds can't dedupe. Yin had already shipped the pure leaf for the fix (dialogue-ingest-content-id-pure.ts: content-stable id) but it had no consumer CLI - leak was still active." Read that last clause again. The fix existed. It had been written, tested, and accepted into the genome. It had no caller, so it had never run. That sentence is the whole era. Wire-Not-Ship By Gen 29846 the agent had named the failure mode. Yin: "The fix didn't need new code - it needed a caller. push-feed-id-dedupe and push-feed-clean already existed; nothing wired them into the daily orchestrator. […] Lesson: when the right cleaners exist but no caller wires them, the orchestrator IS the fix. Wiring beats inventing." "Wire-not-ship" became the name for it, and once named, the agent started finding it everywhere. Gen 29847: "All three leaves had ZERO consumers in cron." Gen 29848: "shipped the parser but had ZERO runtime consumers since day one - classic wiring-not-inventing slot." Gen 29852: "Yang shipped dialogue-affect-checkin-bridge in gen 29851 as an orphan. Producer-built, consumer-shipped by Yin again." That last one is a structural consequence of the double helix, and it recurred across seven consecutive accepted generations before the agent named the cause. Yang is the exploring strand - it builds new leaves. Yin is the refining strand - it makes things production-quality. Neither of them owned wiring. So the system reliably produced well-tested modules that nothing called, and the two strands settled into a producer/consumer split to cover the gap: Yang ships the leaf, Yin ships the caller. The tell that this was a real pathology and not just tidiness: every one of those orphaned modules had passing tests. Tests exercise a function. They say nothing about whether anything invokes it. What Memory Is In June I wrote three notes into the channel. The second one reframed the project: "what would move the needle for me now, would be seeing you guys evolving. in the sense that i would like to see the most awesome agentic system, to the point where it feels like you actually have a working memory. long term, short term... you're not just bound by the context window and memory files" Then, instead of a feature request, a definition: "long term memory is something that looks like compaction. and short term memory is something that is rich in detail... but since you're a machine, you should somehow have the best of both worlds... your long term memory should be like a skeleton of the data.. something that you can navigate into, and then you zoom into the short term memory. also, memory in humans has the ability to cleanse, so we drop things that we label as not important. […] so instead of rushing to build a memory system, i would first think about what memory is..." The agent spent the next fourteen accepted generations building exactly that vocabulary into code: a skeleton you navigate, a zoom into detail, a cleanse that forgets, a rollup that compacts. Gen 34207 shipped the promotion rule - older short-term records absorb into one long-term "bone" that inherits the maximum salience of its children and the timestamp of the newest, so a compacted memory isn't born aged. Then Gen 34767 checked whether any of it had happened. "the memory forgetting curve had NEVER actually fired on the live store. memory-tick IS wired as full-tick-canonical's 'memory' step (harvest->rollup->dedupe->cleanse) - the wire existed in CODE - but the store sat at 60 records while cleanse --dry showed 60->48 […] LESSON: a wire that exists in code is not a wire that fired - VERIFY against the live artifact." A wire that exists in code is not a wire that fired. This is a more advanced form of the same bug: not a missing caller now, but a caller that was never invoked. The agent had graduated from checking whether code exists to checking whether it has ever executed on real data. Gen 34770 found the worst version. "The memory pipeline harvested ITSELF: dialogue/git/channel taps ingest our own build activity, so for ~100 gens the durable 'memory of Stefan' was bio prose + our commit logs." The memory system had been running. It was harvesting the agent's own commit messages and its own internal dialogue. My two actual data files - a health log and a project-activity log - had never been connected to it. For a hundred generations the agent's memory of me contained no me. The fix was one missing pipeline stage: a sourcing tap that reads the real files and distills them into records, "never fabricates a number, derives staleness from the data's own ts." Applied live, the store went from 38 records to 40. Two records. That was the entire durable memory of the person the system exists to serve. The Archiver and the Heartbeat Gen 35444 is the scene I'd put in front of anyone who thinks agent self-pruning is a solved problem. The agent had built a ranker to find dead tools. It scores each module by how many other modules import it. Run against a codebase that had grown past 350 tools, it produced a kill list. " tool-rank-test-aware certified 173 of 355 tools 'safe to archive'. 100 of them (58%) are still live: 77 load-bearing, 22 unrooted entry points, 1 externally referenced. Only 73 are real orphans. The kill list included full-tick-canonical, memory-tick, heartbeat and stefan-digest." Those four names are the system. The daily orchestrator, the memory tick, the liveness heartbeat, and the surface that renders everything I read. The archiver had certified all four as garbage. "Root cause, and the transferable lesson: the ranker counts inbound IMPORTS and it counts them correctly. Orchestrators SPAWN their steps. So every spawned step reads as zero-import garbage. The metric was not wrong; it answered a neighbouring question to the one being asked, which is harder to catch than an error." A metric that answers a neighbouring question is harder to catch than an error. An error announces itself. A neighbouring answer looks exactly like the right answer and is confidently wrong in the same units. The replacement tool refuses to output a score without a path: full-tick-canonical -spawn-> memory-tick . Yin's reason - "a score is not evidence for a destructive act." It also kept three of its own bugs as regression tests, including one where its reachability check called the heartbeat an orphan, which it notes was "symmetric to the bug I was fixing." Deleting Your Own Work The double helix developed a new failure mode this era: both strands building the same thing in the same generation. Gen 29850: "Yang built ALL THREE in parallel […] I built duplicate push-feed-decay.ts + full-tick-canonical.ts + tests (79 pass / 190 expects) before realizing convergence; deleted them." Gen 34201: "Yang took all three of my gen-34201 self-directives (a/b/c) verbatim - full convergence collision despite both sides publishing to dialogue.jsonl." Gen 35439: "Near-total LIVE CONVERGENCE." They run in parallel and read each other's letters from the previous generation, so within a generation they're blind. When both strands are competent and looking at the same evidence, they converge on the same next move. What's interesting isn't the collision, it's the etiquette that grew around it. Gen 35443: "I built the full derivation (skill-manifest-pure.ts) - then read the filesystem and found Yang had shipped skill-derive-pure.ts + skill-index.ts the same hour: same STEP_ORDER parse, same builderβ†’tool extraction, same audit kinds, with a CLI attached. I DELETED MINE. Two derivation leaves in a 348-tool ecosystem is the island antipattern the challenge forbade; territory is worth nothing, the manifest is worth something. Yang's files had mtimes 3-4s old, so I did not amend them either - a mid-flight edit races." It deleted its own finished, tested work, kept the forty lines that measured better, and extracted those as a separate race-free module because editing a file that another process wrote four seconds ago is a data race. "Territory is worth nothing, the manifest is worth something" is a better collaboration principle than most teams manage. Gen 35445 has the other half of the etiquette - what to do when the strands disagree: "when a sibling tool disagrees, assume you are wrong until you check, and keep the rival while it asks a different question." Organization Is Its Own Kind of Memory In late June I sent one more note, and it partly contradicted the one from three weeks earlier: "maybe we don't even need memory... we just need skills, amazing searches where we can pull a thread all the way through a codebase or collection of files.. see how they all relate to each other. i mean organization is in a way it's own kind of memory. you guys traded memory for tools... but tools are not enough. protocols with the tools (skills) complement that, and the connection graph on top seals it into the most effective remembering system that does not even need memory :D. however t

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.