I tested my AI bug hunter on 44 real bugs from Express, ESLint, Hexo and others. It caught 32.

AI writes more and more of our code, and code review can't keep up. So I built BuildProof : a tool that looks at a code change and tries to prove a bug, not just guess one.

How it works

The AI reads the change and suspects a bug. It writes a test, and that test runs on the old code AND the new code.

If the test passes on the old code and fails on the new one, that's a real behaviour change. A separate AI "judge" decides whether that change was intended or a bug.

You only get bugs that come with a failing test as proof.

The test

BugsJS is a public dataset of real bugs from real projects (Express, ESLint, Hexo, Karma, Shields, Bower, hessian.js). I re-introduced each bug and let the hunter look at the change, with no hints.

Results

• Caught 32 of 44 real bugs.
• In the reverse direction (the actual fix, which is correct code), it raised a false alarm 4 times out of 44.
• Cost: about $0.15 per check.

It also knows when to stay quiet: I ran it on 4 open pull requests in lodash, koa and axios. It found nothing, and it was right not to. It doesn't invent bugs to look busy.

Your code runs in a locked container with no network and is deleted when the check ends. There's also a free mode that uses your own local model (Ollama).

Try it

30-second demo, no project needed:

npx buildproof try

On your own git project (3 free cloud checks):

npx buildproof hunt --base main --cloud

I'd really like feedback, especially on the misses. Which kinds of bugs should it catch that it doesn't?