GPT-6 Astra vs Fable 5.1 vs Gemini 3.8 Flash: The Ultimate Comparison
Book: AI That Ships - The series: AI in TypeScript - 5 books, from your first LLM call to agents in production - all five here - My project: Hermes IDE | GitHub - an IDE for developers who ship with Claude Code and other AI coding tools - Me: xgabriel.com | GitHub Three launch tables went up in the same September 2026 week, and every one of them has its own vendor on top. Anthropic shipped Claude Fable 5.1. Google released Gemini 3.8 Flash. OpenAI shipped GPT-6 Astra and called it the start of the AGI era. OpenAI's launch numbers show Astra beating Fable 5.1 on almost every row. Artificial Analysis, which runs its own evaluations, puts Fable 5.1 first on overall intelligence and on its coding agent index. LLM Stats scores the same two models and gets a different ordering again. None of those tables is lying. They measure different things, under different settings, and the disagreement between them is the most useful information in this whole launch cycle. So this comparison does the unglamorous thing: it goes through the numbers, says who measured each one and how, and ends with the only question that matters for your system - which model for which work. The tale of the tape | GPT-6 Astra | Claude Fable 5.1 | Gemini 3.8 Flash | | |---|---|---|---| | Vendor | OpenAI | Anthropic | | | Context window | ~1M tokens | 1M tokens | 1M tokens | | Max output | 128K | 128K | - | | Input price / 1M | $10.00 | $10.00 | $0.75 (promo) | | Output price / 1M | $50.00 | $50.00 | $3.75 (promo) | | Output speed | ~87 tok/s | ~67-69 tok/s | ~305 tok/s | Two prices need footnotes before anything else. Gemini 3.8 Flash's $0.75/$3.75 is promotional through 31 December 2026, after which Google lists $1.50/$7.50 - still a fraction of the other two. Astra also has a fast tier at $20/$100, and Fable 5.1 prices cache reads at $0.25 per million tokens, which matters more than it looks. The speed row is measured throughput from Artificial Analysis, not a vendor claim. Prices move. Check the current pages before you budget: OpenAI, Anthropic, Google. What OpenAI's launch table says Astra's vendor numbers are the loudest, so start there. From OpenAI's launch material, as reported by The New Stack and broken down by Vellum: - FrontierMath Tier 4: 97.6%, against Fable 5.1's 87.8%. - ARC-AGI-3: 99.9%, effectively saturating a fluid-intelligence benchmark that most models struggle to score double digits on. - ExploitBench: 100%, with Fable 5.1 at 70%. - GPQA Diamond: 96.0%. - OSWorld 2.0: 72.6%, at roughly 40 minutes per task - the strongest computer-use score in the launch-week coverage. - AutomationBench: 41.4% vs Fable 5.1's 31.4%. - BenchCAD: 95.9% vs Fable 5.1's 84.3%. - Terminal-Bench 4.0: 57.7% vs Fable 5.1's 55.8%. On research-grade math, abstract reasoning, offensive security and computer use, Astra is ahead of everything, sometimes by a wide margin. Those results are real and worth taking seriously. They also come with footnotes that most of the coverage skipped. Vellum's breakdown lists them. Per that write-up, OpenAI funded FrontierMath's development and has exclusive access to part of the problem set. The ARC-AGI-3 score was produced under OpenAI's own provider adapter harness. The BenchCAD comparison against Claude used modified evaluation settings, and the comparison on ExploitGym, a separate exploit benchmark, ran with time limits removed. And the production model you can actually call is the safety-hardened version, which refuses the advanced cyber work the 100% ExploitBench score was measured on. The numbers are not fake. They are vendor numbers: measured by the party with the most to gain, under settings you cannot fully reproduce. That is exactly the situation where you want a second opinion. What independent measurement says Artificial Analysis runs the same evaluation suite across every major model, independently of the vendors. Its launch-week read: - Intelligence Index: Fable 5.1 scores 66 at maximum effort. Astra scores 61. Gemini 3.8 Flash scores 59 in its high configuration. - Coding Agent Index: Fable 5.1 at ~70, Astra at ~67, Gemini 3.8 Flash at ~61. So the model that loses almost every row of OpenAI's launch table wins the independent aggregate. Both facts hold at once, because they are not measuring the same thing. OpenAI's table leans on frontier academic benchmarks: Tier 4 math, saturated reasoning suites, exploit development. Artificial Analysis weights a broad mix that leans closer to the work most systems do: code, tool use, instructions, long context. Two individual results make the split concrete. On Humanity's Last Exam with tools, Fable 5.1 scores 65.0% against Astra's 57.2% - tool-heavy research is a Fable strength. On DeepSWE v1.1, Astra's 74.1% is barely ahead of Gemini 3.8 Flash's 73.8%, a model costing a thirteenth as much per token. LLM Stats weights things differently again and puts Astra ahead overall, 60.7 to 56.8, while still showing Fable 5.1 ahead on individual agentic rows. Three scoreboards, three orderings. Benchmarks still earn their keep. An aggregate score answers the question "good at what the aggregate weights", and nobody's aggregate weights your workload. When two credible measurements disagree this much, the winner for you lives in the individual rows that resemble your traffic, and in your own eval suite - nowhere else. The economics are the actual differentiator Capability-wise, these three models are closer than any launch table admits. The pricing is not close at all. Astra and Fable 5.1 list identical per-token prices: $10 in, $50 out. Gemini 3.8 Flash lists $0.75 in and $3.75 out until the end of - Run a moderate workload through all three and the shape is obvious. Say 1,000 requests a day, 4,000 input tokens and 800 output tokens each - a back-of-envelope calculation from list prices, no one's measured bill: type Price = { inPerM: number; outPerM: number }; const MODELS: Record = { "gpt-6-astra": { inPerM: 10, outPerM: 50 }, "claude-fable-5-1": { inPerM: 10, outPerM: 50 }, "gemini-3-8-flash": { inPerM: 0.75, outPerM: 3.75 }, }; function dailyUSD(p: Price, reqs: number): number { const inTok = (reqs * 4_000) / 1e6; const outTok = (reqs * 800) / 1e6; return inTok * p.inPerM + outTok * p.outPerM; } for (const [id, price] of Object.entries(MODELS)) { console.log(id, dailyUSD(price, 1_000).toFixed(2)); } gpt-6-astra 80.00 claude-fable-5-1 80.00 gemini-3-8-flash 6.00 Eighty dollars a day against six. Over a month that is roughly $2,400 against $180 for the same request volume. A 5-point gap on an intelligence index has to buy a lot of correctness to justify a 13x bill, and on high-volume routes it usually does not. Between the two premium models the list prices are identical, so the tiebreakers are structural. Fable 5.1's cache reads at $0.25 per million tokens mean a long-running agent that keeps re-reading the same large context pays a fortieth of the input price on the repeated part. Agent loops carry a large stable prefix: system prompt, tool definitions, a repository map. For those, cache pricing is the difference between a context strategy that scales and one that does not. On the other side, Astra's ~87 tokens per second beats Fable 5.1's ~67-69, and Gemini 3.8 Flash's ~305 embarrasses both. If a human is watching the tokens render, that ordering is felt on every single request. Which model for which work Read the individual rows instead of the aggregates and the three models sort themselves with surprisingly little overlap. GPT-6 Astra for research-grade math and science, computer-use agents, and security work - the rows where its lead runs 10 points or more. The 40-minutes-per-task OSWorld result also makes it the current pick for long autonomous desktop tasks, if your harness can handle a call that runs that long. Claude Fable 5.1 for long agentic coding runs and tool-heavy research, where both the independent coding index and the tools-enabled benchmarks put it first, and where cache-read pricing compounds over every loop iteration. If your workload is "an agent working in a repository for an hour", the independent numbers say this is the default. Gemini 3.8 Flash for everything high-volume: classification, extraction, summarisation, chat, and any route where latency is user-visible. It sits a few index points below the other two and costs a thirteenth as much, at four times the throughput. Most of the requests most systems serve belong here, and the DeepSWE result says it is no longer only a small-model tier. Which means the real answer to "which model won the week" is the boring one: you route. Cheap and fast by default, escalate to a premium model on the work shapes where its lead is measured in double digits, and let cost per successful task decide where the boundary sits, not a launch table. The escalation-router pattern is small: call the cheap model, validate the output against something deterministic, escalate on failure, and refuse any request that would blow a per-request cost ceiling. It applies unchanged with these three models in the slots. Every number above is days old, from launch-week material, and some of it will be stale within a quarter: promotional prices expire and models get patched. Treat this post as a snapshot of September 2026 and a method for reading the next launch table. The ranking has a shelf life. The method does not. If this was useful The uncomfortable part of every model launch is that no vendor table, and no independent index, can tell you whether the swap helps your system. An eval suite over your own traffic, cost per successful task on every route, and a router that escalates on signal - that is the machinery that answers it, and it is what AI That Ships builds, in TypeScript, end to end. It is book 5 of AI in TypeScript, a five-book series that runs from your first LLM call through to agents you can leave running in production. Top comments (0)
Comments
No comments yet. Start the discussion.