We compared different LLMs on IMO 2026 [R]

Why IMO Problems?

There are a few reasons why problems from the International Mathematical Olympiad function as a good benchmark for LLMs:

• The problems are new, not included in the training data of any model.
• Hard math problems are quite a good proxy for general intelligence capability.
• These are complex multi-step tasks that can benefit from orchestration / harness engineering.

Results

Frontier models (sol and fable) were able to get perfect / nearly perfect score regardless of harness. For both sonnet and opus, the webapp performance was quite poor, improved by provider harness (claude code) and even further improved using AutoFyn, a customizable multi-agent harness we developed. Even with harness, we were not able to match the performance of the frontier models. Open weight model GLM performed roughly at the same level as sonnet without harness, and improved similarly with AutoFyn.

Numerical scores are available in the attached paper below. Grading was done by a different frontier model as well as manual verification (we are former IMO medalists, able to sanity check the results). There were cases when the model claimed a false solution (on P3 by sonnet, for example), so hallucination issue still persists in a verifiable domain like math.

Hardest Problem

On the hardest problem: P3's key reduction was missed by every sub-frontier model in every harness, including a 20-hour run that proved everything else and stalled at the identical step. The harness supplied retrieval and verification, not a key idea needed for the solution.

Paper: https://github.com/SignalPilot-Labs/AutoFyn/blob/main/results/imo-2026/autofyn-beyond-model-imo26-report.pdf

Audit Trails: https://github.com/SignalPilot-Labs/AutoFyn/tree/main/results/imo-2026