Inside AIEOS: AI Can Write the Spec. It Can't Approve It.

Inside AIEOS: AI Can Write the Spec. It Can't Approve It.

Ask Claude for a system architecture document. You'll get one. It'll have the right sections, confident prose, a diagram, tradeoffs that sound reasonable. It reads like something a staff engineer wrote on a good day.

Now tell me if it's correct. You can't, really. Not by reading it. The thing that makes AI good at producing the document is the same thing that makes the document hard to trust. It generates what's plausible, and plausible is a long way from sound.

I've spent about a year building a framework for that exact problem. It's called AIEOS. This is the first real look at it.

AI is a generation engine, not a decision-maker

That sentence is the whole thing, so I'll sit on it for a second. An LLM is very good at producing a first draft of almost any engineering artifact: a PRD, a test strategy, a release plan, a postmortem. It is not good at deciding whether that draft is good enough to build on. Ask it to check its own work and it'll rationalize. It wrote the thing. It's the last narrator you want grading it.

I made the platform-side version of this argument a few months back, in The Agent Is 20% of the Work. The agent is the easy 20%. The durable system around it is the other 80%, and that's where projects live or die. AIEOS is my answer to what that 80% looks like when AI is writing the specs, designs, and records across a whole software lifecycle.

Three rules that hold it together

AIEOS runs on three rules. They're boring on paper and they matter enormously in practice.

First, separation. The rules for an artifact, the template it fills, the prompt that generates it, and the validator that judges it all live in separate files. Change the prompt and you haven't silently moved the goalposts on what counts as valid. Most AI-writing setups collapse all four into one giant instruction, and then nobody can explain why last week's output passed and this week's doesn't.

Second, freeze before promote. Before any downstream work starts, the upstream artifact is frozen. The architecture document can't quietly shift under the execution plan built on top of it. If it has to change, that's a new version and an explicit re-freeze, not a silent edit. Anyone who's watched requirements move mid-build knows why this rule earns its keep.

Third, validators judge, they don't help. A validator returns PASS or FAIL. It does not offer suggestions. It won't rewrite your draft or meet it halfway. The moment a validator starts helping, it's collaborating with the thing it's supposed to be checking, and you're right back to AI approving its own work.

Fifteen layers, and it's a loop

The work is organized into fifteen layers. The first eight are a sequential pipeline: strategy, product intelligence, solution sourcing, engineering execution, release, reliability, insight, diagnostics. Each one answers a single question and hands a frozen artifact to the next.

The other seven are cross-cutting kits (quality, security, data, infrastructure, documentation, peer review, business process). They fire on a trigger instead of waiting for a fixed turn. Security doesn't queue up behind release. It activates when an artifact event calls for it.

And it's a loop. Layer 7, insight and evolution, feeds what production actually taught you back into Layer 2, where the next round of product decisions gets made. Delivery flows down. Learning loops back up.

1  Strategy
2  Product Intelligence <---------------+
3  Solution Sourcing                     |
4  Engineering Execution                 | what production
5  Release & Exposure                    | taught you
6  Reliability & Resilience              |
7  Insight & Evolution -----------------+
8  Operational Diagnostics

The part that makes it real

A layer model on a whiteboard is worth nothing. For a framework like this, the real test is whether each component does what its contract says it does, especially the adapters that wrap real tools like Semgrep, Trivy, Syft, cosign, and Flux.

AIEOS handles that with conformance attestation. Every adapter has to pass a conformance suite that checks its actual output against a frozen contract. When it passes, CI produces a signed attestation using Sigstore: keyless signing through Fulcio, logged in Rekor's transparency log. No valid attestation for the current contract version, no registration. The adapter doesn't get to play.

Adapter output
    |
    v
Conformance suite
checks real output vs. the frozen contract
    |
 +----+----+
 FAIL      PASS
    |        |
    v        v
Rejected.  Sigstore signs
No         keyless: Fulcio + Rekor
registration.
            |
            v
       Signed attestation
            |
            v
       Registry admits the adapter

I finished wiring that up last week. Every repo in the framework, more than forty of them, now runs CI. All thirteen adapters produce real signed conformance attestations, not a green check that means "the unit tests passed and we're hoping the rest works." The signing loop is verified end to end.

The orchestration core of the agent harness, the routing and state and convergence code, sits at 100% line coverage. Turning the conformance checks back on found real bugs that a permissive CI config had been hiding. An SBOM adapter emitting the wrong CycloneDX schema version. A signing adapter still using a legacy bundle format. The exact kind of thing that looks fine until something finally checks it.

The orchestration is the easy part

The agent harness runs generation across providers. It supports a few routing strategies: fall back to a second provider when the first fails, run providers in sequence, run them in parallel and take a consensus, or route by cost. A convergence loop regenerates an artifact against validator feedback until it passes, or until the loop detects it's oscillating and gives up.

That's what people picture when they hear "multi-agent," and it's real. It's also a few hundred lines. Everything I described before it, the specs and freezes and validators and signed attestations, is the part that actually took a year.

Where to look

AIEOS is open. The code lives under github.com/wtlinnertz. Start with aieos-governance-foundation. That's the root of the whole thing: the layer model, the kit manifest, and the structural rules every other repo inherits.

Fair warning. I built most of this solo, over about a year, and it shows in the uneven places. Some kits are further along than others. But the spine is real and it's tested, which is more than I can say for most things wearing the "AI governance framework" label right now.

One more time, because it's the entire point. AI can write the spec. It can't approve it. The whole framework is what it takes to keep those two jobs in different hands.