Harness Engineering: The Engineering Discipline Behind Reliable AI Agents

Harness Engineering: The Engineering Discipline Behind Reliable AI Agents

We spent decades engineering software to be predictable. Now we're building systems around something that isn't.

For years, the central question in AI engineering was: Which model should we use? GPT vs Claude. Reasoning models vs general-purpose models. Bigger context vs smaller context. One model vs another. But as AI systems evolve from chatbots into agents that can inspect repositories, modify code, run commands, browse the web, interact with APIs, execute tests and work for hours without constant supervision, the question is changing.

The interesting problem is no longer simply how capable the model is. It is: How do we build a system that allows a probabilistic model to do reliable engineering work?

That is the problem space increasingly described as Harness Engineering. The term has emerged as a way of thinking about everything surrounding the model that turns raw intelligence into an operational agent: context, tools, execution environments, permissions, state, orchestration, memory, feedback, verification and recovery.

A useful mental model is:

Agent = Model + Harness

The model provides the intelligence. The harness determines how that intelligence is exposed to the world, constrained, observed, corrected and ultimately turned into useful work. And that distinction is becoming increasingly important.

The Model Is Not the Agent

A language model, by itself, is remarkably limited. It receives information and generates a response. It doesn't inherently have:

• a persistent workspace
• a filesystem
• a Git repository
• a shell
• a browser
• durable state
• permissions
• a test environment
• access to current information
• a mechanism for recovering from failure

Those capabilities come from the system around the model.

A coding agent might look deceptively simple from the outside:

User
  ↓
Model
  ↓
Code

But the actual system looks more like:

┌─────────────────┐
│   Human Intent   │
└────────┬────────┘
         ↓
┌─────────────────┐
│ Context & Rules  │
└────────┬────────┘
         ↓
┌─────────────────┐
│      Model       │
└────────┬────────┘
         ↓
┌─────────────────┐
│ Tools & Runtime  │
└────────┬────────┘
         ↓
┌─────────────────┐
│ Feedback / Evals │
└────────┬────────┘
         ↓
┌─────────────────┐
│ Recovery / Loop  │
└────────┬────────┘
         │
         └──────→ Continue

That entire surrounding system is the harness. The Harness Engineering material makes this distinction explicitly: the harness includes prompts, tools, skills, infrastructure and orchestration-the parts that make the model useful as an agent.

This leads to a surprisingly important conclusion: the model may be the most intelligent component in the system, but it isn't necessarily the component that determines system reliability.

Why Better Models Don't Make Harnesses Obsolete

It is tempting to believe that sufficiently capable models will eventually eliminate the need for all this scaffolding. If the model can reason well enough, why do we need elaborate instructions, verification loops, context management and recovery mechanisms?

There is some truth to this. As models improve, certain pieces of the harness become unnecessary. Anthropic has explicitly described cases where behaviors previously compensated for by the harness improved in newer models, allowing parts of the harness to be removed. But this creates a more interesting dynamic. The harness doesn't disappear. The operating envelope expands. A better model allows us to attempt harder tasks. Harder tasks introduce new failure modes. The harness evolves to handle those failures.

So the progression looks less like:

Better model → Less harness

and more like:

Better model
     ↓
More capable agent
     ↓
More ambitious tasks
     ↓
New failure modes
     ↓
Better harness
     ↓
Even more capable agent

Anthropic's recent harness work makes a similar point: every harness contains assumptions about what a model cannot reliably do, and those assumptions need to be revisited as models improve.

That is why Harness Engineering isn't simply a collection of temporary workarounds. It is becoming an ongoing engineering discipline.

From Prompt Engineering to Context Engineering to Harness Engineering

There is an interesting evolution here.

Prompt Engineering

Initially, the focus was: What should I tell the model? We experimented with wording, examples, roles, instructions and few-shot prompts. Then agents became longer-running and more capable. The question became: What information should the model have at this moment?

That is Context Engineering. Anthropic describes context engineering as the broader problem of curating the information available to a model during inference-including system instructions, tools, MCP servers, external data and conversation history.

And then comes the next question: What environment should the model operate in? That is where Harness Engineering begins. Now we're thinking about:

• What context is available?
• What tools exist?
• What can each tool do?
• What requires approval?
• Where does execution happen?
• What state persists?
• How is work verified?
• What happens when something fails?
• How does the agent continue?
• How do multiple agents collaborate?
• How does the system improve over time?

The progression is therefore:

Prompt → Context → Harness

Each step expands the engineering boundary.

1. Context Is a Finite Engineering Resource

One of the biggest mistakes in agent design is assuming: More context = better results. It doesn't. A model may technically be able to accept a huge amount of information while becoming less effective at using it.

Anthropic describes context as a finite attention resource and recommends optimizing for the smallest set of high-signal information that maximizes the desired outcome.

This changes how we think about agent memory. A mature agent shouldn't simply receive:

Everything we know about the project

It should receive:

What matters for this task
+ what the agent needs to find
+ what it needs to remember

That leads to several important harness primitives: structured repository documentation, persistent memory, plans, skills, just-in-time retrieval, context compaction, durable artifacts.

The Harness Engineering analysis makes a similar distinction between long-lived instructions, persistent memory, session continuity and temporary conversation, emphasizing that context should be governed rather than endlessly accumulated.

This is why context engineering is not simply prompt writing. It is information architecture for an intelligent runtime.

2. The Repository Becomes Part of the Agent's Brain

This is one of the most interesting consequences of agent-first development.

Humans can rely on knowledge that exists outside the repository: conversations, meetings, tribal knowledge, someone's memory, old design discussions. An agent can't. If the relevant knowledge isn't accessible during execution, it effectively doesn't exist for the agent.

OpenAI's experience building an internal product almost entirely through Codex pushed this idea particularly far. They treated repository knowledge as the system of record and deliberately worked to make the codebase itself legible to agents.

This means the repository increasingly contains:

• architecture
• design decisions
• development conventions
• plans
• quality rules
• operational knowledge
• documentation
• skills
• historical context

The repository is no longer just where the code lives. It becomes part of the agent's operating environment.

And this gives us a new architectural quality:

Agent Legibility - A system should not only be understandable to humans. It should be discoverable and understandable by agents.

That means good architecture increasingly has two audiences: human developers and machine collaborators.

3. Tools Turn Intelligence Into Action

A model that generates text can make a bad suggestion. An agent with tools can make a bad change. That is a fundamental difference.

Once an agent can:

Read
  ↓
Modify
  ↓
Execute
  ↓
Observe
  ↓
Modify again

we've moved from generation to execution.

This is why tools aren't merely features of an agent. They are execution interfaces.

The harness needs to determine:

• which tools exist
• what they can access
• when they can be used
• what permissions they require
• whether they can be interrupted
• how their results are recorded

The Harness Engineering material describes tools in exactly this way: managed execution interfaces that need authorization, scheduling, interruption and proper closure, with higher-risk tools receiving stronger governance.

A filesystem read and arbitrary shell execution may both be "tools," but treating them identically would be poor engineering. The principle is simple: Capability should always be accompanied by control.

4. Permissions Become Part of the Agent's Constitution

Traditional applications often treat authorization as an infrastructure concern. For autonomous agents, permissions directly shape behavior.

Consider:

ReadFile

versus:

Bash

The first is primarily observational. The second can potentially delete files, modify repositories, install software, change configurations, access networks, kill processes and trigger external actions.

So a mature harness needs something more sophisticated than:

Tool → Execute

It needs:

Agent requests action
         ↓
Policy evaluation
         ↓
Allow / Deny / Ask
         ↓
Execute
         ↓
Observe
         ↓
Record state

The important insight is that the permission layer isn't merely protecting the user. It is also protecting the agent runtime from its own mistakes.

The Harness Engineering material captures this philosophy well: permission is an organ of the system rather than an accessory feature.

5. A Real Agent Needs a Heartbeat

A chatbot can be modeled as:

Input → Model → Output

A real agent needs a loop. Something closer to:

Input
  ↓
Prepare state
  ↓
Invoke model
  ↓
Consume response
  ↓
Execute tools
  ↓
Observe results
  ↓
Update state
  ↓
Recover if necessary
  ↓
Continue
  ↓
Stop when appropriate

The Harness Engineering analysis calls this the query loop-the heartbeat of the agent system.

The model invocation is therefore only one part of the runtime. The loop owns: state, tool execution, streaming events, interruption, recovery, continuation and stop conditions.

This is an important maturity test. A system that can answer a prompt is a demo. A system that can maintain coherent progress through changing state, failures and tool calls is an agent runtime.

6. Failure Is Not an Edge Case

Traditional software architecture often starts with:

Happy path

and then adds:

Error handling

Agent systems need a different mindset. For long-running agents, failure is part of the main path.

• Context limits will be reached.
• Tools will fail.
• Commands will return unexpected output.
• Networks will break.
• The model will misunderstand something.
• A subagent may fail.
• A hook may loop.
• A recovery mechanism may itself fail.

The Harness Engineering principles explicitly argue that error paths should be designed as first-class paths, with retry limits, circuit breakers and anti-loop mechanisms established at design time.

The goal isn't to build an agent that never fails. That's unrealistic. The goal is to build one that can fail without losing control of the workflow.

7. Recovery Should Preserve Momentum

Imagine an agent has spent an hour understanding the architecture, locating relevant files, creating a plan, implementing half the feature, running tests and discovering a subtle issue. Then its context becomes unusable.

A naïve system says: Start again.

A good harness asks: What useful state can we preserve?

This is where durable artifacts, Git, plans, memory and structured state become extremely valuable. Anthropic's work on long-running agents emphasizes the use of structured artifacts and task decomposition to allow work to continue coherently across sessions.

Recovery isn't simply:

"Try again."

It is:

"Continue from the best known state."

That distinction becomes increasingly important as agent tasks move from minutes to hours and eventually days.

8. Verification Is Where Autonomy Becomes Trustworthy

Perhaps the most dangerous sentence in agentic development is: "The agent says it's done."

Implementation and correctness are different things. An agent can modify the code, run a test, interpret the result incorrectly, explain why the solution is correct and still be wrong.

This is why verification needs to be an explicit stage. Anthropic's recent long-running coding work uses a planner, generator and evaluator architecture, with the evaluator independently examining the resulting application against concrete criteria.

The Harness Engineering analysis similarly emphasizes that verification should be independent rather than simply being the implementer's final self-check.

A useful pattern is:

Research
   ↓
Implementation
   ↓
Verification
   ↓
Failure?
 ┌─┴─┐
Yes No
 ↓   ↓
Fix  Done
 ↓
Verify again

The important thing isn't necessarily having multiple agents. It is separating responsibilities.

9. Multi-Agent Systems Are About Partitioning Uncertainty

There's a tendency to think: More agents = more intelligence. That's not necessarily true. Five agents performing the same task can simply produce five different versions of the same uncertainty.

The more interesting design is:

Research
   ↓
Synthesis
   ↓
Implementation
   ↓
Verification

Each stage has a different responsibility.

The Harness Engineering material describes the value of multi-agent systems in exactly these terms: research, implementation, verification and synthesis become separate responsibility containers, with the coordinator responsible for reconverging distributed knowledge.

This produces an important principle: Parallelism is useful when it creates better boundaries, not merely more output.

And synthesis becomes critical. Research can be distributed. Understanding must reconverge.

10. Quality Needs Both Rules and Feedback

Martin Fowler's treatment of Harness Engineering introduces another useful distinction: feedforward and feedback.

Feedforward - Tell the agent what good behavior looks like before it acts. Examples: architecture rules, repository instructions, coding conventions,