The Runaway Diff: A Token-Budget Postmortem for Coding Agents

The Incident

The task looked trivial on paper: add a rate limiter to a small Python service and update three call sites. I handed it to a coding agent running on MonkeyCode, an open-source project with free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I did what most engineers would do with a ten-million-token allowance: give the agent a generous budget and walk away. Three hours later I returned to a two-thousand-line diff for what should have been a forty-line change. The test suite was green, which made the failure harder to explain, but the agent's log told a clearer story. The same file had been edited fourteen times, and each edit appeared to revert the previous one before adding something new. The agent was oscillating between two designs, and nothing in my setup was designed to notice.

Diagnosis

My first hypothesis was prompt ambiguity, because the task description did leave room for interpretation about where the limiter should live. I rewrote the prompt with explicit constraints, pinned the exact function names, and added a sentence demanding a minimal diff. The second run was faster, but the log showed the same oscillation pattern, which ruled out the prompt as the primary cause.

My second hypothesis was model quality, and I was ready to blame the free tier until I looked at the evidence. The agent's reasoning trace showed that each design change was locally reasonable; the problem was that the agent had no reason to stop exploring. It kept finding marginal improvements, and each one invalidated an assumption from the previous iteration, so the file flipped like a pendulum.

The root cause was not the model and not the prompt; it was the absence of a budget as a first-class constraint. A ten-million-token allowance is generous enough that an agent can treat it as infinite, and my harness gave the agent no termination criterion beyond "finish the task." Without a stopping signal, the agent optimized for an unstated goal: a perfect solution rather than a correct one. The real bug lived in my workflow, which is the most useful kind of bug to find.

The Fix

The fix was a small Python harness that treats token budget the way a test treats an assertion. It caps the run, measures the diff, and fails loudly when the agent exceeds the cap or oscillates. I wrote it in about forty lines, and it is reproducible on any machine with Python and git.

#!/usr/bin/env python3
"""budget_harness.py - cap an agent run and detect edit oscillation."""
import argparse
import subprocess
import sys
import time

def estimate_tokens(text: str) -> int:
    # Heuristic: roughly four characters per token for code and prose.
    return len(text) // 4

def edits_per_file(diff: str) -> dict[str, int]:
    files: dict[str, int] = {}
    current = None
    for line in diff.splitlines():
        if line.startswith("+++ b/"):
            current = line[6:]
            files[current] = 0
        elif current and line.startswith("+") and not line.startswith("+++"):
            files[current] += 1
    return files

def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--task", required=True)
    parser.add_argument("--budget-tokens", type=int, default=150_000)
    parser.add_argument("--max-edits-per-file", type=int, default=8)
    parser.add_argument("--agent-cmd", required=True)
    args = parser.parse_args()

    if estimate_tokens(args.task) > args.budget_tokens:
        sys.exit("Task itself exceeds the token budget.")

    start = time.monotonic()
    proc = subprocess.run(args.agent_cmd, shell=True, capture_output=True, text=True)
    elapsed = time.monotonic() - start
    used = estimate_tokens(proc.stdout + proc.stderr)
    print(f"elapsed={elapsed:.1f}s estimated_tokens={used}")

    if used > args.budget_tokens:
        sys.exit(f"Token budget exceeded: {used} > {args.b