Grok 4.7 was built to work for hours. It still fails most of the time.
The New Stack

Grok 4.7 was built to work for hours. It still fails most of the time.

A coding agent running for hours can make dozens of decisions as it edits files, runs tests, and works through errors. One wrong turn can carry through the rest of the task unless the agent catches it.

Grok 4.7 was built to work for hours. It still fails most of the time.

Training Approach

SpaceXAI appears to be training Grok for exactly that problem. The company released Grok 4.7 on Sunday, and its training approach is uniquely different. SpaceXAI used a longer reinforcement learning run deliberately weighted toward harder tasks, including problems that take "many hours" to complete. The company says that training also made Grok better at verifying its own work and managing longer context.

Every failed approach from an agent adds more history for the model to keep straight, and one bad assumption can follow it through the rest of the task. SpaceXAI is trying to address that with better context management and self-verification, so Grok can catch a wrong turn before it builds on it.

SpaceXAI says the improvements came from pairing the larger base model with an extended reinforcement learning run deliberately shifted toward harder, multi-hour problems, and that the model specifically improved at two capabilities critical to long-horizon execution: self-verification and long-context management.

Endurance Benchmarks

Grok 4.7 scored 38.0% on Terminal-Bench 4.0, up from 20.3% for Grok 4.6. It also improved from 40.4% to 46.3% on CursorBench 4.0, which tests longer-running coding workflows inside the editor, and from 1,546 to 1,657 on AA Briefcase v1.1, an evaluation of multi-hour professional work.

For context, Anthropic's Claude Fable 5.1 scores 57.9% on Terminal-Bench 4.0 according to the independent leaderboard; Grok 4.7 still trails Fable 5.1 here.

What's Still Unknown

An agent working unattended for hours has to keep track of a growing interaction history while checking that each step worked before moving to the next. Those problems surfaced in a recent benchmark of private codebases, where even the best-performing model failed more than 60% of the time.

SpaceXAI says Grok 4.7 improved at both context management and self-verification, although it hasn't explained how. The company did not disclose whether the context gains came from architectural changes, summarization, retrieval, or better retention across long sequences, or how it evaluated self-verification during reinforcement learning.

The Harness Is Becoming Part of the Model

SpaceXAI trained Grok 4.7 to natively understand the Grok Bot harness, bringing the model and the surrounding infrastructure closer together. Agent harnesses handle the work around the model, including exposing tools, formatting terminal responses, feeding execution results back into context, and deciding what happens next.

OpenAI took a similar approach last week when it opened its Codex harness as the Agents API, turning the infrastructure behind long-running agents into a managed service. With Grok 4.7, SpaceXAI is pushing some of that integration into training.

A model already familiar with its harness doesn't have to learn every tool format and interaction pattern through prompting at runtime. That could reduce the overhead involved in tool use and multi-step execution, although SpaceXAI hasn't published enough detail to show how much of Grok 4.7's performance gain comes from harness-specific training.

Training models around specific tool schemas, context formats, and execution environments could make it harder for developers to swap models without sacrificing agent performance. That problem grows as agents take on more of the development cycle.

Google's recent work on making Go easier for AI agents to work with took a different approach, changing the development environment rather than the model. In both cases, the model is no longer the only piece being optimized. The systems around it are changing too.

Where the Gaps Still Are

Grok 4.7 starts at $2 per million input tokens and $6 per million output tokens. At that price, multi-hour agent runs may cost less, but reliability remains an issue. Grok 4.7 scored 38.0% on Terminal-Bench, while Fable 5.1 reached 57.9%.

Read on The New Stack ↗ ← Back to News

Comments

No comments yet. Start the discussion.