probatum - your tests pass. does it actually boot?

The itch

Your test suite is green. 142 tests pass. You tag, you deploy. And the app doesn't boot - because the tests mocked the store, and the real boot replays the WAL, and one segment is missing. No test ever touched that path. That gap - between the tests pass and it actually stands up - is the whole reason probatum exists.

What probatum checks (that tests don't look at)

One file, probatum.yaml. Flat checks, no logic. The curl, the grep, the process supervision are embedded - you just declare the rules that make a check pass or fail.

# start the real service, wait until it answers, keep it alive for what follows
- name: api boots
  run: ./target/debug/myapp --port 8080
  ready: http://127.0.0.1:8080/healthz
  timeout: 15

# embedded curl
- get: http://127.0.0.1:8080/api/version
  expect: 200
  contains: ['"version"']

# embedded grep - only the lines written DURING this run
- name: app log is clean
  log: /var/log/myapp/app.log
  absent: ["ERROR", "panic"]

probatum starts your real system, interrogates it, reads its logs - then kills the whole process tree on every exit path: normal end, probatum's own panic, Ctrl-C, SIGTERM. If probatum dies, nothing it started survives. No zombie port between runs, no orphan server hanging around.

And the contract that every hand-rolled bash script is missing: failed (exit 1) ≠ couldn't observe (exit 2). A missing binary, an unreachable URL, a dirty environment - that's not a bug in your code, that's "I couldn't look." Conflating the two makes you chase ghosts. probatum refuses an already-dirty environment (the port answers before the service starts) instead of destroying it: it never purges what it doesn't own.

The rest is the noise filter: panic, traceback, FATAL caught by default; and humble when unsure - it shows the tail of the output and says "exit 1", it never invents a cause. A false cause is worse than no cause.

Why not just a cidx stage?

The real question. I already maintain cidx, a declarative CI/CD runner: it runs tools against my code (trivy, gitleaks, go-test), identically local and in CI, grouped into phases and pipelines. So why a second tool?

The honest admission first: both can run an arbitrary command. A cidx stage could script the same checks. But they don't verify the same object:

• cidx ──▶ the CODE, statically (which scanners/linters/tests run, local == CI)
• probatum ──▶ the RUNNING SYSTEM, dynamically (boot, readiness, HTTP behavior, log window, teardown) ← cidx has none of that machinery

The sharing rule is one line: run a tool against my sources → cidx stage. Start my system and observe its behavior → probatum. So no merge. probatum joins the roster of tools cidx orchestrates - like trivy, like go-test - linked by a preset.

One source of truth, two launchers:

probatum.yaml ← one file
    │
    ├── inner loop (a dev or an agent, in seconds) : probatum run
    └── outer loop (CI, pipeline) : cidx run test

And the guardrail - the actual design decision: if probatum grows phases and pipelines, it's becoming cidx → refuse. If cidx grows readiness and process ownership, it's becoming probatum → same. The boundary isn't an accident, it's held on purpose. Two small tools that each do one thing and keep their edge.

The proof engine with nothing to prove

Confession: probatum didn't start like this. It was born a "proof engine" - promises, oracles, an evidence registry, the whole vocabulary of formal verification. Impressive on paper. I cut it. That vocabulary was overhead, not value - grandiosity around a simple need: one line, one file, run my checks, show me only what matters. A proof engine that, in the end, had nothing to prove - just something to make useful. (Yes, the pun is cheap. It's also exactly what happened.) Dropping the word "proof" is the same discipline as everywhere in these projects: name the limit, throw out the vocabulary that inflates. A tool earns its words; it doesn't borrow them from theory to look impressive.

The real first reader is the agent

What drove the cut: probatum's first reader isn't a human scanning a terminal. It's an AI agent (and the human over its shoulder). An agent chaining fifty ad-hoc bash runs drowns in output and piles up orphan servers on port 8080. probatum hands it one verdict (probatum run --json) and owns everything it starts - so you can loop it tight without cleaning up behind it. That's the differentiator versus "just run bash": process ownership, the collapsed verdict, and the evidence trail to replay. A misspelled YAML key is an error, never a silently skipped check - because an agent believing it tested something it didn't is the worst lie of all.

Where it actually stands

probatum is twelve days old. v0.1.0, 21 commits, a ~1 MB static binary, a 14 MB image, zero external contributors, zero users besides me. It's not on crates.io - I haven't decided to claim the name yet.

What's solid today:

• the contract is frozen (sources × flat rules, exit 0/1/2, process ownership on every exit path)
• it tests itself - the root probatum.yaml builds, lints, runs the demo end-to-end and asserts the negative scenarios are actually caught
• it runs inside cidx 2.1.0 as a test-phase preset

The two dogfood each other: cidx's pipeline runs inside probatum, probatum ships as a cidx preset. Each has already found bugs in the other.

What's not done: everything else. This is a v0.1.0, and this post is about its boundary, not its maturity.

The ecosystem I'm aiming at

The map, then: cidx orchestrates the outer loop, probatum verifies the running system in the inner loop. They dogfood each other - cidx's pipeline runs inside probatum, probatum ships as a cidx preset - both are built for the agent, and each refuses to become the other.

It's not a platform. It's a toolchain where every link does one thing, stays replaceable, and is honest about its edge. The bet isn't in the size of each tool - it's in the boundaries between them, held on purpose. probatum is one more link: the one that answers, once the tests are green, the only question left - sure, but does it actually stand up?

French original on arcker.org.