I said my tool supported multiple instances. A stranger running 15 of them filed three bugs. All three were right.

I maintain safari-mcp, an MCP server that lets AI agents drive a real Safari session on macOS. Over a few releases this summer I shipped per-session tab isolation - the feature that lets several agents share one browser without stealing each other's tabs. I wrote about it here. Twice. I was reasonably proud of it. Then a stranger showed up with a deployment I had never imagined - three macOS profiles, up to fifteen concurrent server instances - and filed three bug reports in the space of seven seconds. All three were right. The reports What made them remarkable wasn't just the hit rate. It was the form. Each issue had a description of the deployment, a repro, and observed data from their own machines. No speculation, no "it feels flaky." They had read the code, formed a hypothesis about where it breaks at their scale, and then watched it break exactly there. Bug 1: the memory guard that guarded nothing. The server has a WebKit memory monitor that sweeps idle tabs before Safari's content process balloons. It also has a gate: only the instance acting as "extension host" - in practice, whichever instance won the race for port 9224 - was allowed to sweep, and only its own tabs. In a fifteen-instance deployment, that means fourteen instances accumulate tabs forever and one instance politely cleans up after itself. The guard was effectively inert precisely in the topology where memory pressure is worst. The embarrassing part: the gate was redundant. There was already a lock file guaranteeing one sweeper per cycle. Someone (me) had added a second safety mechanism on top of an existing one, and the second one broke the feature. The fix was a deletion. Bug 2: the ownership file that ate its own entries. Instances record which tabs they own in a shared JSON file. The save path was read-at-startup, modify-in-memory, overwrite-on-save. Classic. With one instance, harmless. With fifteen, every save was a snapshot overwrite that silently dropped whatever the other fourteen had written in the meantime. Note what this bug doesn't do: it doesn't crash, it doesn't touch the wrong tab, it doesn't log anything. Losing an ownership entry is fail-safe - a tab just stops being tracked. Which is exactly why it survived. Bugs that fail loudly get fixed in week one. Bugs that fail politely wait for someone to run fifteen instances and notice entries vanishing. The fix was merge-on-write: union the disk state with local state, newest timestamp wins, with explicit removal deltas so deletions don't get resurrected by a stale peer. I considered a file lock and deliberately skipped it - the residual race window is sub-millisecond and fail-safe, and a lock adds a failure mode of its own. That tradeoff is written down in the code, next to the merge. Bug 3: the poll that never backed off. When a profile's Safari window is closed, the server polls for it to reappear - a fixed 3-second interval spawning an osascript subprocess each time. Forever. Their logs showed the consequence: roughly 1,200 log lines and 1,200 subprocess spawns per hour, per idle instance. Multiply by fifteen. The fix is the boring one you already know: self-scheduling backoff, 3s doubling to a 60s cap, reset on success. While fixing it I found a bonus bug the report hadn't mentioned - a trace log being written to __dirname , which for an npm package means inside node_modules. That file had been quietly growing inside an installed package directory the whole time. What I actually learned Your test topology is a claim about the world, and nobody peer-reviews it. I had tested multiple instances - on one profile, launched from one place, in small numbers. The reporter ran three profiles times five. Every one of the three bugs lived precisely in the gap between my topology and theirs. Not one was visible in mine. The tests were green the entire time; the suite passed 78/78 the morning the reports came in. Green tests verify the world you thought to simulate. Fail-safe failures are the long-lived ones. All three bugs share a property: the system kept working. Memory got worse slowly. Entries vanished silently. Subprocesses spawned invisibly. If any of them had thrown an exception, they'd have died in code review. Design your failures to be loud, or accept that the quiet ones will be found by a stranger, years later, at a scale you didn't test. Sometimes the fix is a deletion. The memory-guard bug existed because two safety mechanisms overlapped and the newer one strangled the feature. I didn't need to write cleverer coordination code - I needed to remove the redundant half. The diff that fixed the scariest-sounding of the three bugs was net negative lines. A good bug report at foreign scale is free QA you could not have bought. I could not have afforded to stand up a fifteen-instance, three-profile deployment as a test rig for a feature I wasn't sure anyone used that hard. Someone built it because they needed it, then handed me the findings with repros. The only correct responses are fast verification, honest credit, and shipping the fix. All three fixes went out in one release the next day, each issue closed with a reply documenting not just what changed but which alternatives I rejected and why - including the lock I chose not to add. That last part matters more than it looks. The reporter now knows the merge-on-write has a known, bounded, fail-safe race. The next person who hits something weird there won't start from zero - the reasoning is in the thread, not in my head. The uncomfortable question The feature worked. The story I told about it was, at my scale, true. It took someone else's production topology to reveal that "supports multiple instances" was really "supports the multiple-instance shapes I happened to try." What's the biggest gap between your test topology and a user's real deployment that you've been bitten by - and did the bug fail loudly or politely? Top comments (0)