Evaluating J-space entropy as an error predictor across 7 datasets on Qwen3-4B [R]

Background

Anthropic’s Jacobian Lens work introduced a way to inspect verbalizable representations inside language models. Follow-up experiments suggested that entropy in this internal “workspace” might help identify confidently incorrect answers. I tested that hypothesis on Qwen3-4B across ~11,400 examples from seven distinct datasets, including TriviaQA, PopQA, NQ-Open, TruthfulQA, HotpotQA, GSM8K, and CommonSenseQA.

Three Main Findings

It can complement output confidence on factual retrieval. On datasets such as PopQA, workspace entropy sometimes improved error-routing precision at low review budgets, particularly among answers that were already high-confidence.

It does not reliably detect internalized misconceptions. On TruthfulQA, workspace entropy was substantially weaker than output confidence. Incorrect answers could still have a clean, low-entropy internal representation.

Its calibration is highly task-dependent. A threshold calibrated on TriviaQA failed on GSM8K because correct mathematical reasoning had much higher baseline entropy. Multiple-choice formatting also weakened the signal substantially on CommonSenseQA.

Conclusion

The overall result is narrower than “internal entropy detects hallucinations”: it may be a useful complementary routing signal for confidently incorrect factual answers, but it does not behave like a task-general error detector. This is currently a single-model study, so cross-model validation is the most important next step.

Resources

The repository contains the full methodology, limitations, raw data, metrics, plots, and reproducible notebook:

https://github.com/dasjoms/jspace-hallucination-eval

I‘d be interested in feedback on the experimental design if anyone feels like giving their thoughts. Note: I already posted this to r/LocalLLaMa yesterday but think this might also fit here.