Chaos, then control: the maturity model most AI rollouts skip over
Every engineering org has rolled out AI coding tools by now, but far fewer have actually seen the outcomes they were hoping for.
Letâs say an organization did the obvious things: bought licenses, ran a lunch-and-learn, unleashed their teams on the new tool. Adoption numbers appear impressive. Nearly everyone touches a coding agent every week.
But six months in, they find, the results are wildly uneven. A handful of engineers are doing genuinely different work: taking on projects nobody wouldâve attempted before, shipping things on their own that would once have taken a team. But plenty of employees are treating AI as glorified autocomplete: something that exists to make their daily activities slightly faster and easier, not something with the potential to raise the bar in a big way.
That gap isnât usually diagnosed as a rollout problem. Itâs identified as a talent problem (âI guess some people are just better with these tools than othersâ) or a tooling issue (âIf only we had better toolsâ). But the real problem is that most orgs skip a crucial stage in AI rollouts: the part where chaos is tamed and codified into a system.
A borrowed line, a real sequence
Vivek Raghunathan, SVP of engineering at Snowflake, opened a recent Leaders of Code conversation with a maxim borrowed from Intelâs Andy Grove: During any platform shift, you have to let chaos reign before you can rein in the chaos.
The implication is that these are two distinct phases with two distinct jobs. Jumping into the second one before the first one is finished doesnât save time; it just produces a rollout that performs like a coin flip. Will it work? Who knows!
Raghunathanâs team built their AI rollout as a five-stage progression:
- Let chaos reign.
- Discover what actually works.
- Codify it into a shared language.
- Rein in the chaos by scaling that language across the org.
- Reorganize around the new reality.
Crucially, step 5 cannot take place until the previous four stages have done their work. Most companies, understandably eager to show a return on the tools theyâve bought, compress or skip straight past stages two and three. They mandate a tool, then jump to expecting consistent gains without ever building the connective tissue that turns individual competence into organizational knowledge.
What the skipped stages actually look like
Measuring the wrong thing, too early. Itâs tempting to judge an AI rollout on hard numbers right away: lines of code, PRs merged, tickets closed. Raghunathanâs team set out to avoid this, operating on the assumption that those metrics are easy to game and donât tell you whether people are getting better at using the tools.
In the âlet chaos reignâ phase, the only metric that matters is adoption itself: Are people actually using it? Are they using it for more than one kind of task? Everything else is noise until you know what âgoodâ looks like.
No shared vocabulary for whatâs working. This is the stage most rollouts skip entirely. Once people are experimenting, some of them build (or stumble onto) genuinely better ways of working. But if nobodyâs watching for those successful experiments, they can stay locked in individual heads.
Raghunathanâs team intentionally went looking for them by asking engineers about the best practices theyâd developed in their own work. They ended up with roughly 14 named patterns, including: plan in writing before you write code, fence your agents into separate git worktrees so parallel work doesnât collide, keep an orchestrating agent free of heavy context so it can delegate instead of getting bogged down.
The names matter less than the fact that they exist at all: a pattern language, deliberately modeled on the âGang of Fourâ software design patterns, gives a team something concrete to teach and measure themselves against.
Scaling by mandate instead of by mechanism. Once you have a set of practices that actually work, the instinct is to announce them and expect adoption: a mandate. Raghunathanâs team instead built a mechanism: regular âfocus weeks,â where the entire org sets aside dedicated time to catch up on the paved path or to forge past it.
That approach works because itâs a structural answer to a structural problem: You canât tell people to internalize new habits during the same overloaded sprint cadence you had before AI showed up.
Reorganizing before the previous steps are done. The last stage-changing how roles, responsibilities, and org charts actually work-is the one everyone wants to talk about first, because itâs the most visible and the most dramatic. But it comes last for a reason.
It sounds obvious once you say it, but you canât sensibly redesign a role around AI-assisted work until you know what AI-assisted work reliably looks like at your company. (The proverbial cart preceded by horse comes to mind.) Skipping ahead to this stage usually leads to restructuring around guesses.
Why organizations skip the middle
These steps arenât skipped because business leaders donât understand that adoption does not automatically equal mastery. Itâs because those middle stages are unglamorous and genuinely hard to run.
Discovery doesnât feel like progress. Watching what your best people are doing, then naming and sharing it out, doesnât lend itself to a press release the way âwe rolled out Tool X org-wideâ does. Itâs slower, less visible, and easy to deprioritize when thereâs pressure to show a return on a big tooling investment.
Codifying practice requires someone to go looking. Raghunathan describes actively seeking out the âfearless explorersâ in his org and getting them to explain what theyâve found. It sounds simple, but in practice many organizations stumble here. You need a real time investment from someone senior enough to translate their findings into something teachable. It also feels skippable when everyoneâs plate is already full.
Itâs tempting to treat the tool as the whole answer. Buying licenses is a decision you can make once. Building a pattern language, then a mechanism to spread it, then eventually a new org structure, is an ongoing practice. And itâs much harder to build a practice than it is to make a purchase.
What running the sequence actually looks like
Resist grading the rollout before you understand the range. In the early months, track whether people are using the tools across a range of tasks-writing and reviewing code, drafting design docs-rather than whether output metrics have moved. A wide range of outcomes at this stage isnât a failure signal; itâs raw material you need to inform stage two.
Go looking for whatâs already working, on purpose. Somewhere in your org, a handful of people have already figured out something non-obvious. Find them, and get specific about what theyâre actually doing differently. That specificity is what turns a one-off into a pattern.
Give the pattern language a name and a distribution mechanism. A list of best practices in an unread wiki wonât spread throughout the org. To do that, you need a dedicated block of time, whether itâs a recurring practice session or a focus week like Raghunathanâs team used, where people are expected to learn the paved path or do some trailblazing of their own.
Hold off on the org chart. The instinct to redesign roles and responsibilities around AI is understandable, but keep in mind that theyâre the payoff of the first four stages, not a shortcut past them. An org chart built on last quarterâs guesses about how AI-assisted work would go is going to need rebuilding anyway.
The step you canât skip
Remember the team from the intro: near-universal adoption, wildly uneven results, no clear understanding of why. The tools arenât the problem, and neither are the individual team members. The absence of a middle stage-the unglamorous work of discovering, naming, and deliberately spreading what actually works-is the real problem.
Skipping the middle stage produces the gap between, on one hand, engineers who are much more productive and creative with AI on board, and on the other, folks using a more sophisticated version of autocomplete.
Letting chaos reign is the easy part. Organizations do it by mandating a tool and stepping back. Reining in that chaos is the part that takes actual management: finding whatâs working, giving it a name, and building a mechanism to spread it before you reach for the org chart.
If your rollout has adoption numbers but no shared vocabulary for what âgoodâ looks like, which stage are you actually in-and is anyone responsible for getting you to the next one?
Comments
No comments yet. Start the discussion.