IDKMesh: What if AI agents had to prove their work?

The problem

A few days ago I wrote about one result from IDKMesh: a panel of 25 verifiers that, after measuring their shared errors, was effectively worth only one independent verifier. That experiment is interesting, but it starts in the middle of the story. So this post is the simpler version: What is IDKMesh, why am I building it, and how is it supposed to work?

The problem is that AI agents are getting very good at producing things. They can:

• write code,
• edit files,
• call tools,
• generate tests,
• open pull requests,
• and sometimes operate infrastructure.

But there is a basic problem: An agent saying “I finished the task” is not the same as evidence that the task was done correctly. If we scale from one agent to ten agents, or one hundred agents, generation becomes faster. Verification can become the bottleneck. And if all of those agents or reviewers make similar mistakes, adding more of them may only give us more confidence, not more truth.

That is the problem IDKMesh is exploring.

What is IDKMesh?

IDKMesh is an open-source research and engineering project about coordinating:

• humans,
• AI agents,
• software tools,
• verifiers,
• and different kinds of compute around uncertain work.

The name comes from a simple idea: I don’t know. You don’t know. Together, the mesh can discover, build, verify, and learn.

For me, the most important word is not “mesh.” It is verify. IDKMesh tries to separate the thing that does the work from the thing that decides whether the work should be trusted.

A simple example

Imagine I give an AI agent this task: Fix a bug in a Python project.

A normal agent workflow might look like this:

Task ↓
AI agent ↓
Code change ↓
“I done”

That is useful, but the same system that created the answer is also telling us the answer is good.

IDKMesh is aiming for something more like this:

Human goal ↓
Bounded Work Unit ↓
One or more replaceable workers ↓
Candidate results ↓
Independent verification ↓
Recorded evidence ↓
Human / governance decision

The worker can propose. The verifier can check. The evidence can be inspected later. And the final authority stays separate.

Worker success != acceptance.

That small rule changes a lot.

How it works

At a high level, IDKMesh uses a few simple building blocks.

1. Give the worker a bounded job

   Instead of giving an agent unlimited authority over a project, work is described as a Work Unit. The task says what needs to be done, what the boundaries are, and what security constraints apply.

2. Let workers produce candidates

   A worker is allowed to try. There can be multiple attempts. The first answer does not need to become the final answer. Workers are replaceable. Their output is a candidate.

3. Verify separately

   A verifier should not simply trust the worker’s own completion claim. Verification has its own plan and produces its own result. This is where tests, independent reviewers, policy checks, or other evidence can enter.

4. Keep the evidence

   IDKMesh treats provenance and verification results as real artifacts, not temporary chat messages. The goal is to be able to ask:

   • What was the task?
   • Who or what produced this result?
   • What checked it?
   • Which evidence supported the decision?
   • Can we replay or inspect it later?

5. Keep the final decision explicit

   A worker does not get merge authority just because it says “success.” A verifier recommendation is also not automatically the final decision. The integration step stays explicit.

[Place the IDKMesh explainer image here.]

What can you actually use today?

This part is important because I do not want to describe research as a finished product. The full Verified Swarm Runner is not yet a production-ready multi-agent platform. The repository is still a research and engineering foundation.

But one useful tool is already available: idkmesh gate-audit. It takes verdicts from a review panel you already have and asks a practical question: How many independent votes is this panel actually worth?

The repository also contains:

• experiments,
• schemas,
• validators,
• simulations,
• interoperability work,
• and pieces of the future runner.

Try it

If you have Python 3.11+:

git clone https://github.com/MSKazemi/idkmesh
cd idkmesh
python -m venv .venv
source .venv/bin/activate
pip install .
idkmesh gate-audit examples/gate-audit/panel-votes.example.json --pretty

You can also run the repository demo: python scripts/demo.py. No model API key is required for that demo.

What I want IDKMesh to become

The long-term idea is bigger than a review tool. I want to explore whether large groups of humans and AI agents can work on real problems without making trust an afterthought.

That means learning how to:

• divide work into bounded tasks,
• run multiple independent attempts,
• use different workers and tools,
• measure whether verification is actually independent,
• keep reproducible evidence,
• learn from failures,
• and scale without giving autonomous workers unlimited authority.

I do not know yet how far this model can scale. That uncertainty is part of the project. The repository is designed to test those questions rather than pretend they are already solved.

Why open source?

Because a system about verification should itself be inspectable. If IDKMesh makes a claim, I want the experiment, code, result, or limitation to be visible. And I especially want other people to find the assumptions that are wrong.

If you are interested in AI agents, verification, distributed systems, CI, developer tooling, or human-in-the-loop systems, take a look at MSKazemi/idkmesh on GitHub. You do not need to understand the whole project to contribute. Even showing where the newcomer path is confusing is useful evidence.

Suggested DEV tags

• #ai
• #opensource
• #python
• #agents

This article was prepared with AI assistance and manually reviewed against the current IDKMesh repository before publication.