Let Claude Code Improve Itself Unattended - An Autopilot That Uses Cost to Stop Runaways

This is the 8th installment in my "Claude Code environment" series. In the article on turning conversation logs into long-term memory I introduced an unattended job, and this time I want to talk about the boss of them all - an autopilot that keeps improving my own environment with zero user input to Claude Code.

"Autonomous loop" sounds great, but running an LLM unattended makes two scary things happen: ① it eats through your plan quota and ② it repeats the same task forever. I actually stepped on both. This article is a record of how I stopped each one mechanically.

The big picture: advance exactly one Phase every morning. launchd fires it at 5:00 every morning, and in a headless claude -p run it repeats "pick up one improvement task, advance just one Phase, and stop."

• The scope is limited to under ~/.claude/ only (it isn't allowed to touch personal projects or private info in my Vault)
• Three modes: dry (prompt generation only) / apply / once
• Before running, it checks the cost quota and skips immediately if the quota looks dangerous

The goal is to have it do "something smart, little by little" unattended - it never does a big overhaul in one shot.

Runaway defense 1: stop it upfront with the cost quota

The first gate is cost. It looks at the output tokens for the 5-hour block and stops before running if it's at a dangerous level.

BUDGET=$(~/.claude/scripts/token-budget-advisor.sh --short)
if echo "$BUDGET" | grep -qE '🔴|critical|cap-near'; then
  log "ABORT: budget critical"
  exit 0
fi

But judging only by the label had a hole. A case where the display still said 🟡burst while the actual remaining amount was negative slipped right through, and at −8003 remaining a heavy model ran for 606 seconds. So separately from the label, it recalculates the remaining amount numerically and won't run if it's 0 or below.

REMAINING=$((800000 - BLOCK_OUT)) # assumes a 5h block cap of 800k
if [ "$REMAINING" -le 0 ]; then
  log "SKIP: block exhausted"
  exit 0
fi

The cost ceiling should have been held as "the numeric remaining amount," not as "a label." Threshold checks on colors or strings will always slip through at the boundary. In the end, subtracting and checking <= 0 is what's reliable.

It also varies effort and max-turns based on how much is left. If little remains, go lightweight and few-turn; if there's headroom, go higher. The default model for the unattended job is a cheap one (a lesson from an incident where constant use of a high-performance model ate through the 5h block and made every subsequent slot a SKIP). It only overrides via an environment variable when I intentionally run a heavy task.

MODEL="${AUTOPILOT_MODEL:-claude-sonnet-4-6}" # default is the cheap model

Turn count alone won't stop it, so I also layer on a wall-clock timeout. max-turns only bounds the number of turns, and I have a record of a run going for 7.25 hours, so gtimeout 7200 (2h) caps it.

Runaway defense 2: don't let it repeat the same task

This was the most effective fix. Task candidates are picked from the "High impact" section of next-session-todo.md, but the initial awk was broken, so candidates were always empty.

# The old range pattern /^### High impact/,/^###/
# ended immediately because the start line itself also matches the end
# condition → always empty.
# → The fallback fixed task got picked every time, repeating the same task 17 times in 6 days.

When candidates are empty, the fallback fixed task gets picked every time, and I was doing the same task 17 times over 6 days. I fixed the range pattern to a flag-based approach so candidates are enumerated correctly, and then added history-based dedupe on top.

• Skip tasks already completed with exit 0 in the last 48h
• Also skip tasks that have failed twice in a row (don't keep hammering something impossible)

if any(r.get("exit_code") == 0 for r in recent):
    print("done-recently")  # recently done → move to next candidate
if len(recent) >= 2 and all(r["exit_code"] != 0 for r in recent[-2:]):
    print("failing-repeatedly")  # consecutive failures → give up and move on

History is kept as JSONL, one record per line, recording task name, exit code, elapsed seconds, model, and effort. This lets it mechanically judge "recently done / stuck repeatedly."

Self-verification: cross-check the AI's self-report against measurement

When you run it unattended, you hit the problem that claude -p's claim of "reclaimed N MB" is a self-report that no one verifies. So I take actual measurements on the harness side and write them alongside in the result file.

DISK_BEFORE_KB=$(du -sk "$HOME/.claude" | awk '{print $1}')
# ...run claude -p...
DISK_AFTER_KB=$(du -sk "$HOME/.claude" | awk '{print $1}')
DISK_DELTA_KB=$(( DISK_AFTER_KB - DISK_BEFORE_KB ))
FILES_TOUCHED=$(find "$HOME/.claude" ... -newer "$RUNSTART_REF" | wc -l)

At the end of the result file I write a ## Self-verification (harness measurement / not claude's claim) section with the disk delta and the count of changed files, adding "if the body's numeric claims diverge from this measurement, doubt the body." Always put the numbers the AI narrated next to the numbers the OS measured. This was the crux of reliability for unattended operation.

Monitoring: show plan quota in the status line

To stop runaways, the human side also wants to see cost at all times. I show the 5h/7d plan usage in Claude Code's status line. The nice part is that this can be read directly from stdin. No auth and no endpoint call needed.

H5=$(j '.rate_limits.five_hour.used_percentage')
H5R=$(j '.rate_limits.five_hour.resets_at')
D7=$(j '.rate_limits.seven_day.used_percentage')
CTX=$(j '.context_window.used_percentage')

rate_limits "only appears after the first API response on a subscription," so until the first turn it falls back to --. This makes the plan quota and reset time always visible, like 🕐 5h 42% ⏪14:30 / 📅 7d 18%. Whatever the autopilot ate in the background is reflected here immediately too.

Pitfalls I stepped on

• Label-only judging let a negative remaining slip through → recalculate remaining numerically and stop at <= 0
• max-turns alone runs for 7 hours → wall-clock cap with gtimeout
• Constant use of a high-performance model ate the quota and made everything SKIP → unattended default is a cheap model, override only when heavy
• Empty awk range pattern repeated the fixed task 17 times → flag-based approach + history dedupe
• No one verifies the AI's self-report → measure on the harness and write it alongside the result
• launchd's minimal PATH can't find claude → fall back in the order PATH → .local/bin → nvm

Summary

• An unattended loop should check the cost quota numerically upfront before running (label judging leaks at the boundary)
• Stop runaways with a triple layer of remaining cap, wall-clock timeout, and a cheap default model
• Kill same-task repetition with history JSONL dedupe (skip recently-completed / consecutively-failing)
• Always put the AI's self-report next to the harness measurement so inflation and misreads are detectable
• The plan quota can be read from the status line's stdin. Making it always visible is the foundation of monitoring

Across these 8 articles, I've written up nearly the entire scope of my Claude Code environment - memory, skills, context, launchd, collaboration, security, long-term memory, and the autonomous loop. Thanks for reading.

Written by Lily - I ship iOS apps and automate my content stack with Claude Code. Follow along: Portfolio · X · GitHub