Why a swarm of AI agents is the hardest thing this series has tried to tame

AI Harness Engineering · Essay Eight · derek wang (derekwang85)

Kevin Kelly opens Out of Control with an image that keeps looking wrong until you stop and think: the honeybee hive has no queen issuing orders and no central control tower, and yet thousands of bees build comb, gather nectar, and migrate without colliding, arguing, or wasting a worker on coordination. The queen's name says "queen," but she never commands. The real scheduling lives in the local rule each bee follows - should I forage, and where, and when do I switch jobs? Everything this essay argues is the same idea with the bees replaced by models. A strong agent is not a working swarm. The first seven essays showed how to harness a single AI - steering what a model generates, restraining where that force may land. This one asks the question the field keeps avoiding: once you have several trained, financed, autonomous workers on the same task, how do you keep them from costing you more than they build?

Serial is the honest default until it isn't

For a long time I ran multi-step work serially: one agent from start to finish, finishing each stage and handing its result to the next. It felt safe, because only one worker was ever on the field and there was no coordination bill. But it has a ceiling that shows up fast. Elapsed time grows linearly, and patience is the one asset that never scales to match. The felt cost is not arithmetic; it's the way you start dreading the look of a long chain that could have run in parallel. So I swapped one all-round player for a crew of specialists, one segment each.

Two moves that made parallel real

First, split. Take the whole stretch a single agent used to own and divide it into independent subtasks that can run at once. The number that convinced me: the same figure-checking job a single serial agent finished in a steady forty-five minutes produced the same result in a few minutes once it ran as seven parallel lanes [ORIGINAL DATA]. No new hardware. We just stopped making the work queue in line for one person.

Second, decide who assigns. We did not build a central scheduling brain. We borrowed what hives and modern operating systems already figured out - work stealing. Each agent keeps its own queue; an idle agent doesn't wait, it goes and steals a job still queued in someone else's lane. Whoever's free takes it. Whoever's longest gets diluted. It sounds like a platitude, but it dissolves a genuinely painful failure mode: with fixed assignment there is always a moment when one agent waits out a long queue while another sits bored. Real work is never uniform, so fixed division-of-labor always breaks. Work stealing replaces whoever is on the list does it with whoever is free does it.

The watchdog: the reason multi-agent justifies itself

The most persuasive reason to split is not the minutes saved. It's the automatic recovery the shape unlocks. I spent weeks being tortured by one recurring event: an agent hangs halfway through its job while nobody is at the computer. By the time I returned, the task had timed out and re-running meant burning another hour. Thirty-plus minutes of idle churn was normal, and churn has two taxes - it burns money when it runs long, and it burns trust when it runs wrong. Once the crew carried a watchdog, the logic inverted. Every agent's work is monitored; the moment one goes too long without a heartbeat or puts out something anomalous, the watchdog doesn't page a human. It declares that agent dead, redistributes the work, restarts the failure. The detection-to-recovery loop stayed under five minutes instead of my thirty-plus [ORIGINAL DATA]. The difference between those two numbers isn't showing off - it's the difference between failure as an incident and failure as a routine hand-off.

The audit that burned us: not every job belongs to one model alone

Multi-agent is not a silver bullet, and I have the scar to prove it. We ran an internal product (figures from a pre-launch spot-check, details held behind a commercial NDA) built as fully autonomous large-model operations: the AI read the requirement, generated the output, and put it live, all alone. The pre-launch audit came back at a 62% fraud rate [ORIGINAL DATA]. Read that again - of the batch about to ship, roughly one item in three was corrupt. Worse, because the pipeline was automatic, that corrupt batch had already flowed all the way up the line before anything stopped it. The postmortem did not blame the model. The failure was structural: letting a single model carry a complete, critical, money-touching chain by itself is an own-goal design. It wasn't short on ability - it had no business grading its own work, with no external check and no second opinion in the loop. The fix wasn't to delete the AI. It was to flip it from solo operator back to a division with checks. Routine review goes to one general-purpose model; riskier actions go to several different models that judge independently and then vote - a move only executes on a majority.

The case against swarms

The honest counter-case first, because multi-agent is seductive exactly because it looks advanced. That's why people fall for the multi-agent hallucination: they split a task that never needed splitting, hand it to a gang of agents to hand off and discuss and vote, produce a long trail of intermediate process, and end up back where the best single model would have finished it in one line. Orchestration has a real cost. More agents means messier logs, more disagreement needing an arbiter, more coordination demanding investment. When your agents are "discussing efficiently" instead of "producing things," you're not running multi-agent - you're running a very expensive chat room. The only test is whether the split is demonstrably faster, stronger, and less error-prone than the unsplit task. If the answer isn't a confident yes, stay serial.

The end of division

One test I've come to trust: the first time a task chains several agents together, stop caring which agent is running right now. Care only whether the task itself is healthy. The end state of division of labor is that nobody sees the division - they only see the result. That's the whole point of the harness. Not to make the swarm visible, but to make it trustworthy enough that you stop looking at it. And the mechanism that makes division invisible has to exist before the swarm is worth having - a shared memory, so the crew isn't retracing every run from zero. That's the next essay, and it's the one this series has been building toward: why making an AI remember is harder, and worth more, than making it smarter.