Mass-Producing Long-Form X Articles With a 5% Human Check: Only the 9/10s Get Through

Why I Didn't Go Fully Automatic

When you let AI write your posts, what you get is an endless stream of articles that look fine but leave nothing behind once you've finished reading. This is part of my ongoing series on building automation infrastructure for shipping personal projects at scale with Claude Code. This time I want to talk about the "5% human check" gate I designed to keep quality high while mass-producing long-form X (formerly Twitter) Articles with Claude Code.

When you let AI write, you quickly end up mass-producing articles that look fine but leave nothing behind once you've finished reading. My honest take is that what determines the post-read impression isn't the accuracy of the information, but whether there's any firsthand information in it.

At first I aimed for a simple three-stage pipeline: generate → quality check → auto-post to X. Wire it all together in shell scripts and run it on cron. Technically it isn't hard. But when I actually ran it, all it did was quietly post uniform 7-out-of-10 articles. You could learn something from them, but they were faceless - "who even wrote this?" It didn't work as something published under Lily's name.

The cause was clear. Claude is good at "aiming for the average score." It tends to produce accurate but stimulation-free articles that offend no one.

The conclusion I reached: to ship a 9/10 article, a human has to hold the final gate. So I settled on a design where Claude mass-produces, and a human touches only 5% to pick out the 9/10s. I gave up on full automation and focused on compressing human judgment down to the minimum amount of work.

The Factory at a Glance

Here's the directory structure of ~/dev/x-article-factory/.

x-article-factory/
├── generate.sh          # Have Claude draft, then self-score
├── review-gate.sh       # Human approves/rejects in a TUI
├── daily.sh             # launchd wrapper (drafts 2 per day)
├── knowledge/
│   ├── persona.md       # Lily's persona and CTA
│   ├── style.md         # Voice and tempo rules
│   ├── post-types.md    # Usable formats (How-to / failure story / before-after, etc.)
│   ├── quality-rubric.md# Criteria for the 10 self-scoring items
│   └── hooks.md         # Collection of opening-hook patterns
├── drafts/              # Drafts that passed the quality gate
├── approved/            # Human-approved articles (awaiting posting)
├── rejected/            # Human-rejected articles
└── logs/
    ├── gen.log          # Generation log (OK / REJECTED_LOWSCORE)
    ├── posted-types.log # History of formats used (last 3 used for dedup)
    └── posted-topics.log# History of topics used (last 8 used for dedup)

The processing flow has three stages.

[launchd] daily.sh
    ↓
Loop N articles (default 2)
    generate.sh
        ↓
    Claude drafts → self-scores (only avg ≥ 7.0 passes)
    drafts/YYYY-MM-DD_HHMMSS.md
        ↓
bash review-gate.sh (human decides in a TUI)
    approved/ ← only the 9s get through
    rejected/ ← even 7-8s get dropped if they don't land
        ↓
Manually paste into X's scheduled posts

The division of labor is simple: AI generates and self-scores, the human just makes the final call.

generate.sh: Claude Writes and Grades Itself

The heart of generation is generate.sh. Let's walk through the design in order.

Config values you can tune via environment variables

MODEL="${X_ARTICLE_MODEL:-sonnet}"
GEN_TIMEOUT="${X_ARTICLE_GEN_TIMEOUT:-420}"
MIN_AVG="${X_ARTICLE_MIN_AVG:-7.0}"

The model defaults to sonnet, the timeout to 420 seconds (7 minutes), and the quality threshold to 7.0. Drafting long-form articles takes time to generate, so I set the timeout with plenty of headroom. Everything can be overridden via environment variables, so experiments like "raise the threshold to narrow down quality" or "try a different model" can be toggled with a single line in .env.

There's also a mechanism where the script exits immediately if a PAUSED file exists.

[ -f "$DIR/PAUSED" ] && { echo "[gen] PAUSED。停止中。" ; exit 0 ; }

If things go off the rails or unexpected articles start coming out, I can stop everything with just touch ~/dev/x-article-factory/PAUSED.

Splitting the knowledge into 5 files and stacking it into the prompt

KNOW=""
for f in persona hooks style post-types quality-rubric ; do
  KNOW+=$'\n\n===== '"$f"' =====\n'"$(cat "$DIR/knowledge/$f.md")"
done

I manage the instruction set for Claude across five files. Here's the role of each.

Rather than one giant prompt, splitting management across files made fine tuning easier - "update only style.md" or "change only the CTA wording."

Grounding: Feed in firsthand information to create distance from AI mass-produced junk

This is the single most important point of the factory design.

GROUND=""
for src in "$HOME/.remember/recent.md" "$HOME/.remember/now.md" \
    "$HOME/Documents/claude-obsidian/wiki/hot.md" ; do
  [ -f "$src" ] && GROUND+=$'\n\n---'"$(basename "$src")"$'---\n'"$(head -c 4000 "$src")"
done
[ -z "$GROUND" ] && GROUND="（作業ログ無し。一般的なClaude Code/個人開発の知見で書く）"

I load three sources as the "firsthand information" passed to Claude.

• ~/.remember/recent.md - recent work notes
• ~/.remember/now.md - what I'm working on right now
• ~/Documents/claude-obsidian/wiki/hot.md - the hot summary from Obsidian (about 500 words)

Each is read up to 4,000 bytes with head -c 4000 and passed into the prompt. The instruction in the prompt is: "Always pick up proper nouns, numbers, and failures and put them into the body. Don't fall back on borrowed generalities." That two-part combination does the work.

For example, if grounding contains information like "the iOS app passed review," "an error came up in a certain step of a script," or "the impression count of a feature I shipped this week," Claude will try to make that the protagonist of the article. This is where the distance from AI mass-produced junk is created.

When all files are empty, it falls back to "no work log" and writes based on general knowledge. Quality drops, so the habit of updating your work log before generating an article is important.

Pulling recent formats and topics from history to avoid duplication

LAST_TYPES=$(tail -3 "$TYPES_LOG" | paste -sd '、' -)
[ -z "$LAST_TYPES" ] && LAST_TYPES="（まだ無し）"
LAST_TOPICS=$(tail -8 "$TOPICS_LOG" | paste -sd '、' -)
[ -z "$LAST_TOPICS" ] && LAST_TOPICS="（まだ無し）"

I pass Claude the last 3 formats and the last 8 topics. By pairing this with instructions like "avoid these and choose a different format" and "avoid rehashing; take a different angle," I prevent runs of articles with the same pattern. The reason topics get a wider window (8) than formats is that topic exhaustion comes first. There are more than 10 formats, but the "Lily lane" of topics (Claude Code / personal dev / AI automation) is narrow.

Output format: a strict structure of meta info + body

I make the output format specification for Claude strict.

1行目: TYPE: <使った型の名前（post-types.mdの番号と名前）>
2行目: TOPIC: <この記事のテーマを15字以内で>
3行目: SCORE: {"hook":N,"value":N,"specificity":N,"firsthand":N,"tempo":N,"reproducibility":N,"structure":N,"surprise":N,"style":N,"cta":N,"avg":N.N}
4行目以降: 記事本文のMarkdown（1行目は「# タイトル」）

The SCORE JSON contains 10 items. Claude scores the article it wrote itself along 10 axes.

Quality gate: avg < 7.0 never reaches drafts

AVG=$(printf '%s' "$SCORE" | python3 "$DIR/lib/parse_avg.py" 2>/dev/null)
[ -z "$AVG" ] && AVG=0
if python3 -c "import sys; sys.exit(0 if float('$AVG') >= float('$MIN_AVG') else 1)" 2>/dev/null ; then
  :
else
  echo "[gen] 品質スコア avg=$AVG < $MIN_AVG。棄却（drafts に出さない）。topic=$TOPIC" >&2
  echo "$(date '+%F %T') REJECTED_LOWSCORE avg=$AVG $TOPIC" >> "$DIR/logs/gen.log"
  exit 2
fi

It extracts avg from the SCORE JSON and rejects anything below 7.0. When rejected, it doesn't output to drafts/ and records it in the log as REJECTED_LOWSCORE. The design intent is: "if Claude self-scores below avg 7.0, it never reaches human eyes in the first place." This drops the human review effort for 7-ish articles to zero. Looking at the logs, 20-30% of drafted articles get rejected at this gate.

Note: "Passing 7.0 = a good article" is not true. Claude's self-scoring tends to be lenient, and when you actually read them, there are plenty of "7.2 but doesn't land" articles. The 7.0 gate is a filter to knock out the obvious failures. Picking the 9/10 articles is the human's job.

Anything that passes the quality gate is written to drafts/.

TS=$(date +%Y-%m-%d_%H%M%S)
FNAME="$DIR/drafts/${TS}.md"
{
  printf -- '<!-- TYPE: %s | TOPIC: %s | SCORE_AVG: %s | gen: %s -->\n\n' "$TYPE" "$TOPIC" "$AVG" "$TS"
  printf '%s\n' "$BODY"
} > "$FNAME"

The filename is a timestamp (2026-06-26_142030.md format). The first line embeds meta info as an HTML comment. By displaying this meta line in review-gate.sh, the "format / topic / score" is visible at a glance during review.

Validation and 3 retries

resp_is_valid() {
  local r="$1"
  [ -z "$r" ] && return 1
  printf '%s' "$r" | grep -q '^TYPE:' || return 1
  printf '%s' "$r" | grep -q '^SCORE:' || return 1
  printf '%s' "$r" | grep -qiE 'request timed out|usage limit|rate limit' && return 1
  [ "$(printf '%s' "$r" | wc -c | tr -d ' ')" -lt 600 ] && return 1
  return 0
}

This is the validation function for the generation result. It checks for the presence of the TYPE line, the presence of the SCORE line, the absence of timeout/rate-limit errors, and a minimum body size of 600 bytes. If any of these is missing, it's treated as invalid and retried up to 3 times.

for attempt in 1 2 3 ; do
  RESP=$(timeout "$GEN_TIMEOUT" "$CLAUDE" -p "$PROMPT" --allowedTools WebSearch --model "$MODEL" </dev/null 2>/dev/null)
  resp_is_valid "$RESP" && break
  echo "[gen] 生成失敗(試行 ${attempt}/3)。再試行…" >&2
  RESP=""
done

I add --allowedTools WebSearch so that Claude can use WebSearch to "check what's current right now." The prompt instructs it to "check what's timely with WebSearch if needed," and it kicks in when picking up timely material.

review-gate.sh: The TUI Where the Human Touches Only 5%

The human reviews the articles in drafts/ that passed the quality gate, one at a time, in a TUI.

echo "未チェックの下書き: ${#drafts[@]}本"
echo "操作: [a]承認 [s]保留 [d]却下 [e]編集 [q]終了"

There are five choices.

The preview shows the meta line (format / topic / score) plus the first 60 lines of the body.

head -1 "$f" | sed 's/<!-- *//; s/ *-->//'  # メタ行（型/テーマ/スコア）
tail -n +2 "$f" | sed '/./,$!d' | head -60

The key point of the TUI is that it's designed so you don't have to read the whole thing. The intent is that the first-60-lines preview lets you intuitively judge "approve or not." Even if the score is 7.5 or higher, if reading it feels "not very Lily" or "the firsthand info is basically generalities," I hit d and reject it immediately. Conversely, even at a score of 7.2, if I feel "this episode lands," I sometimes open the editor with e, make a small fix, and approve.

The output after review ends is simple.

echo "承認済み: $(ls "$DIR"/approved/*.md 2>/dev/null | wc -l | tr -d ' ')本（approved/）"
echo "次: 承認分をXの予約投稿に貼る → 投稿したら approved/ から消す"

I manually paste approved articles into X's scheduled posts, and delete them from approved/ after posting. By running the approve → post → delete cycle, queue management is fully handled with just ls approved/.

As a realistic sense of the pass rate, approving 1-2 articles out of every 10 generated (10-20%) is realistic. I run it on the hypothesis that "posting 1-2 nine-out-of-tens beats posting ten seven-out-of-tens for overall engagement."

daily.sh: Drafting 2 Articles a Day via launchd

N="${X_ARTICLE_DAILY_N:-2}"

# nvm default の PATH を通す（launchd最小環境対策）
export NVM_DIR="${NVM_DIR:-$HOME/.nvm}"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" > /dev/null 2>&1

[ -f "$DIR/PAUSED" ] && { echo "[daily] PAUSED" ; exit 0 ; }

ok=0
for i in $(seq 1 "$N") ; do
  if bash "$DIR/generate.sh" ; then
    ok=$((ok+1))
  fi
  sleep 2
done

echo "[daily] $(date '+%F %T') 起草 $ok / $N 本 → 人間チェック: bash review-gate.sh"

This is the launchd wrapper script. By default it drafts 2 articles a day (changeable via X_ARTICLE_DAILY_N). What matters is nvm PATH resolution. launchd's minimal environment doesn't have nvm, and in its bare state you get claude: command not found. Explicitly sourcing nvm.sh resolves it. If you forget this, you end up in the confusing situation where it works in an interactive shell but fails only via launchd.

The sleep 2 in between is a stability measure. Rather than firing off N articles in parallel, waiting 2 seconds before firing the next one makes the API calls more stable. Inside the loop, it watches generate.sh's exit code to