Fable 5.1 vs GPT-6 Astra vs Gemini 3.8 Flash on Real-SWE

If you want the highest first-try success rate on real enterprise code, Fable 5.1 running inside Claude Code wins: it resolved 38.8% of tasks on the Real-SWE benchmark at an estimated $6.96 per rollout, the most expensive setup measured (Specific Labs).

For cost-sensitive teams, Gemini 3.8 Flash in Gemini CLI is the value pick at 31.2% for an estimated $2.50 per rollout, and GPT-6 Astra in Codex CLI splits the difference at 33.8% for $4.67 (Specific Labs).

One honest caveat: the confidence intervals in that top cluster overlap, with Fable at roughly [32%, 45%], Astra at [27%, 40%] and Gemini at [25%, 38%], so read first versus third as directionally consistent rather than settled.

TL;DR

• Fable 5.1 via Claude Code leads Real-SWE at 38.8% resolution, followed by GPT-6 Astra via Codex CLI at 33.8% and Gemini 3.8 Flash via Gemini CLI at 31.2% (Specific Labs).
• Estimated cost per rollout ranges from $2.50 for Gemini 3.8 Flash to $6.96 for Fable 5.1 (Specific Labs).
• Every score is a model plus harness combination, not a model in isolation, which is why the same CLI produced 33.8% with GPT-6 Astra and 16.2% with GPT-5.6 Sol (Specific Labs).
• The tasks are hard: six of the ten sit below 15% resolution, and one, the analytics stream reducer, was never solved by any tested combination (Specific Labs).
• Letting agents run longer did not help. Rollouts under ten minutes failed 71.4% of the time against 73.4% for runs of ten minutes or more (Superpower Daily).
• Missed requirements, not broken syntax, is the dominant failure mode across the field (Specific Labs).
• Last verified: 2026-09-14.

Which LLM is best for coding on real enterprise code?

On the evidence available today, Fable 5.1 paired with Claude Code.

Real-SWE, published in September 2026 by Specific Labs, evaluates frontier models on private production codebases rather than public repositories, with tasks licensed from real companies covering billing, tax, customer migration and infrastructure work (Specific Labs).

Across eight model-and-harness configurations and ten tasks, the benchmark scored 640 rollouts, reporting resolution rate as pass@1 averaged over eight independent runs per task with 95% confidence intervals (Specific Labs).

The full leaderboard, with estimated cost per rollout:

Rank	Model and harness	Resolution rate	Est. cost per rollout
1	Fable 5.1 via Claude Code	38.8%	$6.96
2	GPT-6 Astra via Codex CLI	33.8%	$4.67
3	Gemini 3.8 Flash via Gemini CLI	31.2%	$2.50
4	GLM 5.3 via Claude Code	28.8%	$5.12
=5	Grok 4.6 via Grok Build	23.8%	$3.44
=5	Muse Spark 1.3 via Muse Code	23.8%	$2.74
7	Kimi K3 via Kimi Code	18.8%	$3.90
8	GPT-5.6 Sol via Codex CLI	16.2%	$2.65

All figures from Specific Labs.

Usage data was incomplete for Grok 4.6 and Kimi K3, so their real costs may be higher. The cost spread is corroborated independently by LavX News.

If you are comparing the two flagship contenders directly, our earlier breakdown of GPT-6 Astra against Claude Fable 5.1 covers pricing and behaviour outside benchmark conditions.

Why is the benchmark leader also the most expensive?

Because Fable 5.1 writes far more.

On the tax jurisdiction task it produced a mean of 78,000 output tokens per rollout, 95,000