DEV Community

I released TraceMotive v0.1, got roasted, and rebuilt the biggest problems in v0.2

A few days ago, I released TraceMotive v0.1. TraceMotive is an open-source, local-first tool for tracing and debugging AI agent execution. The first version worked. But after releasing it, I got some pretty harsh feedback - and most of it was fair. The biggest problems were: - traces disappeared when the Collector restarted - trying the UI required multiple terminals and Node/npm - it felt more like a trace viewer than a debugger So instead of adding more frameworks or bigger features, I made those problems the scope of v0.2. What changed in v0.2 TraceMotive v0.2 now has persistent local SQLite storage. If you restart TraceMotive, your traces are still there. The normal setup is also much simpler now: pip install "tracemotive[server]" tracemotive serve Then open: http://127.0.0.1:8765 The production UI is packaged with TraceMotive, so normal users no longer need Node, npm, Vite, or a separate frontend dev server. The other big addition is trace-to-trace comparison. You can select two agent runs in the UI and compare things like status, errors, latency, token usage, model/request parameters, tool inputs/outputs when captured, and spans that only exist in one run. There is also a Changed only filter and side-by-side field comparison. One problem I found while building comparison Repeated tool calls looked simple at first. If the same tool appeared multiple times, I thought I could just match the first call with the first call, the second with the second, and so on. That turned out to be a bad assumption. Before shipping the production comparison API, I built an evaluation set and tested what happened when repeated tool calls were inserted, removed, or reordered. The naive ordinal approach produced incorrect matches: inserted call: 4 incorrect matches removed call: 2 incorrect matches reordered calls: 4 incorrect matches The problem was that the matcher could look completely confident while pairing the wrong logical calls. So I changed the production behavior. If repeated calls cannot be deterministically paired, TraceMotive does not guess. It reports them as an ambiguous group instead of pretending they are exact matches. After the change, the production regression results became: insertion: 0 false exact matches removal: 0 false exact matches reorder: 0 false exact matches I may experiment with suggested or heuristic matches later, but if I do, I want them clearly separated from exact matches. Still local-first The privacy model from v0.1 is still there. Content capture is disabled by default, redaction happens before events enter the transport queue, and tracemotive serve binds only to 127.0.0.1 . TraceMotive also does not send its own analytics or telemetry to an external service. v0.2 does not claim to provide automatic root-cause analysis, causal proof, replay, or first-divergence detection. For now, the goal is simpler: keep the traces, make the debugger easy to start, and make differences between two executions easy to inspect. GitHub: https://github.com/doraemonfv-glitch/tracemotive If you build AI agents, I'd be interested to hear what still feels annoying, confusing, or missing. Concrete failure cases are especially useful right now. Top comments (0)

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.