Can a Cheap Model Beat a Frontier Model? Rebuilding Recursive Language Models with Codex

Large language models have enormous context windows now. That does not mean they use all of that context reliably. As prompts grow, models can miss details, lose track of relationships, or produce plausible summaries instead of doing the exhaustive work a question requires. The Recursive Language Models (RLM) paper proposes a different interface: keep the large context outside the model, expose it as a variable in a persistent programming environment, and let the model inspect, partition, and recursively query smaller pieces. We rebuilt that method with an unusual constraint: - no OPENAI_API_KEY ; - Codex CLI as the model backend; - gpt-5.4-mini for both the RLM root and every subcall; - a direct frontier model only as a separate baseline. The result was encouraging, expensive, and more nuanced than “cheap model equals frontier model.” What an RLM changes A normal model call looks roughly like this: large prompt -> model -> answer An RLM instead gives the root model metadata about the input and a Python REPL containing the real context: question | root model | persistent REPL holding the context |-- inspect and search with code |-- split context into useful chunks |-- call smaller LMs over those chunks |-- validate and aggregate results `-- return the final answer The important detail is that the root model does not need to carry every document, record, tool result, and partial answer in its own context window. Large intermediate values can remain in REPL variables. Subcalls receive focused, locally understandable tasks. That makes RLM less like a bigger prompt and more like an out-of-core data-processing system whose semantic operator happens to be a language model. What we actually tested We used an OOLONG trec_coarse validation example from the protocol described in the RLM work. The input was a 308,367-character context containing 3,182 general-knowledge questions. Each question implicitly belonged to one of six answer types: - numeric value - entity - human being - location - abbreviation - description and abstract concept The labels were not present in the context. The task was to infer the labels and identify the least-common category. We compared: - A direct gpt-5.6-sol Codex call. - An RLM where the root and all leaf calls were locked to gpt-5.4-mini . The direct frontier call answered abbreviation and scored zero. The mini-only RLM answered numeric value , matching the gold answer. | Method | Result | Model calls | Elapsed time | |---|---|---|---| | Direct frontier call | Incorrect | 1 | 40.1 seconds | RLM with gpt-5.4-mini only | Correct | At least 238 | 6,120.3 seconds | The RLM root first inspected the structure of the context. It then classified chunks, retried malformed responses, reduced the chunk size, reclassified all 3,182 questions using structured JSON outputs, checked that it had coverage, and calculated the minimum. This is exactly the sort of work that a direct model call often approximates but a recursive program can force itself to perform. The uncomfortable but useful audit Getting the final answer right did not mean every intermediate judgment was right. We compared the mini model's inferred counts against the validated labels: | Label | True count | Mini inferred | |---|---|---| | Numeric value | 398 | 402 | | Entity | 521 | 623 | | Human being | 544 | 488 | | Location | 571 | 493 | | Abbreviation | 571 | 560 | | Description and abstract concept | 577 | 616 | The model made substantial row-level classification errors. It still found the correct minimum because numeric value had a 123-item margin over the next-smallest true category. That distinction matters. This run shows that decomposition changed the outcome and allowed a cheap model to solve one problem that the direct frontier call missed. It does not prove that the cheap model reconstructed the data exactly, and one row does not establish general equality between the two systems. The honest claim is narrower: On suitable long-context tasks, a cheap model inside an RLM can match or outperform a direct frontier-model call. Use cases already demonstrated by RLM research The paper evaluates four useful task shapes: 1. Semantic aggregation OOLONG requires labeling and aggregating information spread throughout a large input. Real applications include: - customer-feedback analysis; - support-ticket taxonomies; - survey aggregation; - incident and application-log analysis; - quality-control statistics over text records. Our experiment belongs to this category. 2. Multi-document research BrowseComp-Plus requires joining evidence across documents in a very large offline corpus. Analogous applications include: - literature reviews; - technical-documentation research; - contract and policy comparison; - due-diligence document rooms; - evidence-backed competitive research. 3. Repository-scale understanding The paper includes LongBench-v2 CodeQA, where questions require reasoning across files in a codebase. Probable uses include: - architecture mapping; - migration-impact analysis; - dependency and license audits; - security triage; - locating missing tests; - comparing implementation against documentation. 4. Cross-record and pairwise reasoning OOLONG-Pairs asks the system to construct relationships between combinations of records. Applications could include: - entity resolution; - policy-conflict detection; - matching candidates against constraints; - finding related incidents; - identifying incompatible configurations; - relationship discovery across an archive. These workloads can grow quadratically, so they need strict budgets and deterministic post-processing. A practical new use case: making sense of agent-session archives While exploring our local Claude Code history, we found a single session transcript that was 242 MB and contained 39,570 JSONL records. All project transcripts together occupied about 3.6 GB. The large session was not 242 MB of useful conversation: - about 176 MB was attachment records; - about 29 MB was assistant events; - about 20 MB was user and tool-result events; - about 12 MB was file-history snapshots. This is an excellent RLM-shaped problem. A deterministic first pass can stream the JSONL, hash duplicate attachments, reconstruct parent-child event relationships, merge subagent logs, and extract messages, commands, file changes, tests, commits, errors, and outcomes. An RLM can then analyze normalized episodes and recursively build: - a cross-session project timeline; - a decision register; - a map of attempted and abandoned approaches; - recurring failure patterns; - unresolved tasks; - evidence-linked summaries of what actually shipped. The final report should cite session IDs, event IDs, timestamps, commands, and Git commits. Otherwise, it is merely another plausible summary. Other probable use cases The same decomposition pattern should transfer to: - long incident timelines assembled from logs, tickets, and chat; - scientific evidence extraction across papers and experiment records; - compliance control-to-evidence mapping; - large archives of meetings, email, or project documents; - ranking records against a nuanced rubric; - graph filtering and multi-hop relationship discovery; - reconciling conflicting claims across many sources; - constructing structured datasets from heterogeneous text. The recurring requirement is not simply “the input is long.” A good RLM task has four properties: - The context can be partitioned or searched programmatically. - Smaller semantic subtasks remain understandable to the cheap model. - Intermediate results can be stored in a structured form. - The final result can be verified or recomputed. Where RLM is probably the wrong tool RLM is a poor default for: - low-latency chat; - simple questions that fit comfortably in one prompt; - sparse retrieval where grep or conventional search is sufficient; - creative writing that depends on a single coherent voice; - exact high-stakes decisions without an independent verifier; - public execution of untrusted model-generated Python; - high-volume synchronous APIs with tight latency budgets. Our successful row took roughly 102 minutes. That is acceptable for a research run or an overnight audit, not for an interactive endpoint. What a reusable package should look like The useful abstraction is not an OOLONG runner and not one universal prompt. It is a context-compute runtime with a small set of reusable recipes: run( context, objective, recipe, answer_schema, verifier, budget ) -> answer + evidence + validation + trajectory + usage Initial recipes could include: aggregate_records evidence_synthesis repository_analysis cross_record_join timeline candidate_ranking For our intended configuration, the Codex backend would keep both root and subcalls locked to gpt-5.4-mini . A frontier model would appear only in evaluation runs, never inside the RLM call tree. Production use would also require an isolated execution environment, call and token limits, schema validation, redaction, prompt-injection defenses, resumable runs, and source-level evidence for every important claim. What comes next The one-row result is a proof of mechanism, not a benchmark victory. The immediate research questions are: - Does the advantage survive across all 50 paired OOLONG tasks? - Can concurrency reduce the 102-minute runtime without changing quality? - Which decomposition recipes transfer cleanly between domains? - How much verification is required for exact row-level work? - Can a mini-only RLM turn multi-gigabyte agent histories into a reliable, source-linked development narrative? RLMs do not magically turn a cheap model into a frontier model. They change the computation available to that model. Sometimes that difference is enough to turn a wrong one-shot answer into a correct, auditable process. That is a more interesting result than the slogan. Top comments (0)