How to Build an AI Software Factory: Agents That Open, Review, and Merge PRs

How to Build an AI Software Factory: Agents That Open, Review, and Merge PRs

TL;DR

An AI software factory is five stages with a gate at each one. The agent is the cheap part.

The short version: every company that made this work built the gates before the fleet. Spotify's Fleetshift shipped in 2023, two years before it had an agent to put in it. Generation scales with spend, review does not, and that asymmetry is the whole design problem.

Where Firecrawl Fits

Agents need live web context the repo does not carry. Search + scrape for the open web and a curated developer index for code, both behind one MCP block, with prompt injection detection on every fetch.

On January 6, 2026, Stephen Toub opened nine pull requests from his phone at 35,000 feet. Seven of them merged. He works on dotnet/runtime , and he wrote up what the experience told him:

AI changes the economics of code production. One person with good judgment and a phone can generate PRs faster than a team can review them. That sentence is the entire subject. A single engineer with a coding agent can now saturate a team's review capacity from an airplane seat.

The interesting question stopped being how to make agents write code and became how to absorb the output. An AI software factory is the answer companies have converged on, and it is what turns autonomous coding agents from a demo into throughput a team can absorb.

What is an AI Software Factory?

An AI software factory is the system around a coding agent rather than the agent itself. Work arrives from a queue, agents run in isolated workspaces, verification happens automatically, and a human sits at an explicit merge gate. Also called an agentic software factory.

The useful distinction is between an agent and a software factory. Running a coding agent on your laptop is an agent: you choose the task, you watch it work, you read the diff, you merge. Everything except the typing is still you, and your attention is the limit. A software factory moves those steps into infrastructure.

Nobody decides which issue an agent picks up, because intake rules do. Nobody sets up a workspace, because isolation is provisioned. Nobody checks whether the change compiles, because verification runs before a human is involved at all. The person shows up at the end, on the decisions that carry accountability.

Addy Osmani puts it more compactly:

A software factory is harnessing loops at scale. The loops he means are the ones we covered in our guide to loop engineering: an agent that runs, checks its own work, and runs again until a verifier says stop. The software factory is the machinery that runs many of them at once without anyone watching.

Three Properties of a Real Software Factory

• It is queue-driven, not prompt-driven. Work enters from issues, alerts, or a Slack channel, and the system decides what is worth starting. Nobody is typing prompts.
• Environments are disposable. Every agent gets a clean workspace it can destroy, so a bad run costs nothing and parallel runs cannot corrupt each other.
• Verification runs before review. By the time a diff reaches a person, it has already compiled, passed tests, and been checked for scope.

Miss the third and you have not built a software factory. You have built a machine that generates review work faster than you can absorb it, which is the failure mode the rest of this article is organized around avoiding.

The Five Stages of a Software Factory

Most of these systems are built on background coding agents, meaning agents that run unattended in their own environment rather than in your editor. Read enough of these architectures and the same skeleton appears, whatever the company calls it. Mastra ships it as six named stages in Mastra Factory. Spotify describes it as nested feedback loops. Stripe calls the pieces blueprints. The shape is the same.

The stage order matters more than the tooling. Each gate stops work from reaching the next stage, and the expensive stages sit at the end.

Stage 1: How Work Reaches an Agent

Intake decides what is worth starting. Get this wrong and every downstream stage burns tokens on work that should never have begun. The naive version assigns an agent to every open issue. The published versions all filter first.

Sentry's Seer scores each incoming error for actionability and only investigates the ones that clear the bar. Shopify made a different call and routed intake through Slack, with one rule: agents work in public channels, never DMs.

Every conversation is therefore searchable. Anyone at Shopify can jump in.

Stage 2: Pick an Isolation Model

Two agents editing one working directory is the fastest way to lose a day. This is where most homegrown software factories stall, because the obvious answer works until roughly the fourth concurrent agent.

Three models, in ascending order of cost and capability:

Worktrees are where to start. A git worktree is a second working directory on its own branch, sharing one repository.

Stage 3: Give the Agent Hands, Not Just a Brain

A model with a repository is an autocomplete. A model with your test runner, your telemetry, your feature flags, and your deploy tooling is a colleague.

The gap between those two is the tool layer, and it is the least glamorous and most decisive stage in the software factory.

The published numbers say how seriously the leaders take it. Stripe's Toolshed hosts roughly 500 internal tools behind one MCP server, with controls that block destructive actions. Cloudflare runs an internal MCP Portal and generated AGENTS.md across more than 3,900 repositories.

Ramp wires tests, telemetry queries, feature flags, and screenshot verification into every sandbox.

A Representative Tool Layer for a Software Factory