Six Days of a Silent Crash Loop: One Command That Health-Checks 26 launchd Jobs

Losing your job costs you a paycheck. What I didn't expect was to spend that same afternoon discovering that half the automation propping up my side income had quietly stopped running - and nobody, including me, had noticed. Why this setup works Once your personal automation crosses about 20 jobs, "silent death" becomes routine. Human attention has a ceiling. With 3 launchd jobs, you can eyeball them every morning. At 10, it becomes weekly. At 26 - you stop checking altogether. Days go by on a vague feeling that "it's probably running." And when you finally notice, it's been dead for three weeks. That is exactly the reality I ran into right after the layoff. Back when my side income was ¥600,000/month, the machinery holding it up was dying quietly. Nobody noticed, because there was still a salary coming in. It took revenue hitting zero for the question to surface: "wait, how long has that job been down?" launchd breaks quietly. macOS launchd (the daemon-management layer) responds to a crashing script by saying nothing and waiting for the next scheduled run. Even if it's returning exit 78, nobody finds out unless you run launchctl list yourself. In my own environment, com.shun.agentmemory sat in a crash loop for six days in June 2026. Worse was the half-alive state: the port was open but no worker was behind it - every HTTP request returned 404 while the process still existed. The usual liveness check (just confirming the process exists) misses this completely. Experiences like that are why automation-health.sh is designed not just to check, but to fix what it finds, immediately. It's the environment, not the task Automation failures aren't like task failures. When a task fails, somebody gets angry. When your environment rots, nobody gets angry. Your productivity just slowly drains away. What supports ¥1.2M/month isn't any single script - it's the "environment" where all of them stay interlocked and running. skill-harvest generates auto-skills, conversation logs accumulate into the Knowledge Base, that syncs into the Obsidian Vault, agentmemory shares memory across Claude instances. Break one link in that chain and, weeks later, you get a vague sense that "Claude's suggestions have felt thin lately." Understanding why comes even later. What automation-health.sh inspects is precisely each link in that chain. Not just the scheduled launchd jobs, but hook script permissions, skill-harvest log freshness, the last update of conversation logs, the Obsidian Vault's auto-update marker, agentmemory's HTTP reachability - all of it swept in one command, exiting 1 if even one item is RED. [開発者] bash ~/.claude/scripts/automation-health.sh ↓ exit 0 → ALL GREEN / WARN exit 1 → RED あり → StopHookが捕捉 That is: run the script, and exit 0 means ALL GREEN or WARN only, while exit 1 means at least one RED - which the StopHook picks up. The exit-1 design is the important part. Wire this script into the StopHook that fires when a Claude session ends, and "verify automation health every time I close Claude" becomes the default. Put it in cron and you verify it every morning. Neither depends on you deciding to check. The overall flow automation-health.sh inspects nine sections in order. Every section uses the same output style: ✓ 緑 → 正常 ⚠ 黄 → 警告（致命的ではないがケア必要） ✗ 赤 → 失敗（exit 1の原因になる） Green ✓ means healthy, yellow ⚠ means a warning (not fatal, but needs care), and red ✗ means failure (the thing that causes exit 1). Here's the whole structure as an ASCII diagram: bash automation-health.sh │ ├─ [1] launchd ジョブ (com.shun.* / com.lily.) │ 全plistをループ → launchctl list で照合 │ 未ロード？ → launchctl bootstrap で即自動再投入 │ 再投入失敗 → ✗ RED │ 前回 exit ≠ 0 → ✗ RED │ ├─ [2] hooks (8本のシェルスクリプト) │ pre_git_guard / pre_secrets_check / pre_env_guard │ post_audit_log / post_format / post_tsc_check │ stop_notify / user_prompt_submit │ 不在 → ✗ RED / 実行権限なし → ⚠ WARN │ ├─ [3] skill-harvest │ .harvest.log の最終更新が48h以内か │ ├─ [4] 会話ログ (Stop hook → 長期記憶) │ ~/Documents/my-knowledge-base/raw/conversations/ │ INDEX.md の鮮度が24h以内か │ ├─ [5] Obsidian Vault 連携 │ hot.md の自動更新マーカー存在確認 │ index.md のカバレッジ（実ファイル数と記載ページ数の一致） │ ├─ [5.5] agentmemory サーバ │ launchctl 状態 AND http://localhost:3111/agentmemory/health → 200 │ どちらかが欠けていても ✗ RED │ ├─ [6] remember 記憶層 │ now.md / recent.md / archive.md の存在確認 │ now.md の重複バーストを検知（consolidate 遅延の兆候） │ ├─ [7] ディスク / 残骸 │ ~/.claude 実効サイズ（>5GB で ✗） │ security_warnings_state_.json の残骸数 │ ├─ [8] 週次/月次バッチ (7本) │ ログファイルの最終更新時刻 vs 許容時間予算 │ └─ [9] cron ↔ launchd 重複 同一スクリプトが両系統に登録されていないかを確認 （移行後の二重実行バグを防ぐ） ↓ fail > 0 → exit 1（RED あり） warn > 0 → exit 0（致命的問題なし） fail = 0, warn = 0 → exit 0（ALL GREEN） In short: [1] loops every plist and cross-checks launchctl list , auto-bootstrapping anything not loaded; [2] checks 8 hook shell scripts for existence and the executable bit; [3] checks that .harvest.log was updated within 48h; [4] checks conversation-log INDEX.md freshness within 24h; [5] checks the Obsidian Vault's auto-update marker and index coverage; [5.5] checks agentmemory via launchd state and an HTTP 200; [6] checks the remember memory layers and duplicate bursts in now.md ; [7] checks disk size and leftover junk; [8] checks 7 weekly/monthly batch jobs against a time budget; [9] checks for the same script registered in both cron and launchd. Any failure means exit 1; warnings alone still exit 0. Section [1] is the core - automatic self-healing The most important piece is the implementation of section [1]. Writing a script that "checks and reports" is easy. But that leaves the human chore of "see RED, fix it by hand." The launchd section of automation-health.sh re-bootstraps an unloaded job the moment it finds one: # 全 com.shun.* / com.lily.* plist を監視。未ロードを見つけたら冪等に自動再bootstrap # （これが無いと、ジョブがlaunchdから外れてもサイレントに発火しなくなる） uid_num=$(id -u) for plist in "$HOME_DIR"/Library/LaunchAgents/com.shun..plist \ "$HOME_DIR"/Library/LaunchAgents/com.lily..plist; do [ -e "$plist" ] || continue job=$(basename "$plist" .plist) line=$(launchctl list 2>/dev/null | grep -E "\b${job}\b") if [ -z "$line" ]; then if launchctl bootstrap "gui/${uid_num}" "$plist" 2>/dev/null; then ok "$job: 未ロード → 自動で再ロードした" else ng "$job: 未ロード・再ロード失敗 (手動 launchctl bootstrap 要)" fi else exitc=$(echo "$line" | awk '{print $2}') if [ "$exitc" = "0" ] || [ "$exitc" = "-" ]; then ok "$job: ロード済 / last exit=$exitc" else ng "$job: last exit=$exitc (前回失敗)" fi fi done The second column of launchctl list is the exit code. 0 is a clean exit, - means "currently running or never started yet," and any other number is a failed previous run. Jobs quietly returning exit 78 (configuration error) or exit 1 (in-script error) are all caught by this single loop. When a job isn't loaded, launchctl bootstrap gui/${uid_num} re-registers it immediately. The gui/${uid_num} target specifier is the key part - it's the current API from macOS 10.15 onward (the old launchctl load is deprecated). Only when the re-bootstrap itself fails does it record ng (RED) and move on to the next check. Section [5.5] - why a process check alone isn't enough There's a reason the agentmemory check is two-stage. The reason is left in a comment in the code itself: # 2026-06-11 監査の教訓: launchd の exit 78 クラッシュループが6日間誰にも気づかれず、 # さらに「ポートは開くが worker 不在で全API 404」の半生状態は死活監視では見えない。 # launchd 状態 + /agentmemory/health の HTTP 200 の両方を見る。 The actual check logic looks like this: am_code=$(curl -s -o /dev/null -w '%{http_code}' -m 3 \ http://localhost:3111/agentmemory/health 2>/dev/null || echo 000) if [ "$am_code" = "200" ]; then ok "稼働中 (pid=$am_pid / health 200)" elif [ "$am_code" = "000" ]; then ng "プロセスは居るが port 3111 無応答" else ng "port 3111 は開くが /agentmemory/health=$am_code - worker 不在の半生状態" fi By checking the HTTP status code as well, you can detect the half-alive state where the process is up but the API is dead. The same idea generalizes far beyond agentmemory - web servers, AI model API proxies, anything that serves HTTP. Section [8] - time-based liveness checks Liveness of weekly and monthly batches is judged by "when was the log last written." Each of the 7 batch jobs gets its own time budget: declare -a CRON_JOBS=( "weekly cleanup-misc:~/.claude/logs/cleanup-misc.log:192" # 週次→8日許容 "weekly env-audit:~/.claude/logs/env-audit-latest.md:192" "monthly plugin-purge:~/.claude/logs/plugin-purge.log:744" # 月次→31日 "weekly plugin-auto-disable:~/.claude/logs/plugin-auto-disable.log:192" "weekly dotfiles-snapshot:~/.claude/logs/dotfiles-snapshot.log:192" "weekly agents-index:~/.claude/logs/agents-index.log:192" "daily plugin-usage:~/.claude/scripts/plugin-audit-latest.md:48" # 毎日→2日 ) Weekly batches allow 192 hours (8 days), monthly 744 hours (31 days), and daily 48 hours. If a log's mtime exceeds the budget, it's reported as ⚠ WARN . The only evidence that a job actually fired is a write to its log, so looking at the log is the most reliable signal. As a concrete example, here's part of the plist for com.shun.plugin-auto-disable : StartCalendarInterval Hour 6 Minute 45 Weekday 0 It runs plugin-auto-disable.sh apply every Sunday at 06:45 and appends to ~/.claude/logs/plugin-auto-disable.log . Even if this plist is loaded into launchd correctly, the log stops if the script itself exits with an error. Section [1] verifies the load state, section [8] verifies log freshness - only with both layers can you say the job is actually running. Implementation details Why use launchctl bootstrap - avoiding the "old API" macOS launchctl has two APIs, old and new. The old one is launchctl load ; the new one is launchctl bootstrap . launchctl load has been deprecated since around Catalina. It sometimes still works, but it prints nothing useful to the log and behavior can change after a reboot. That's why section [1] of automation-health.sh uses bootstrap : uid_num=$(id -u) if launchctl bootstrap "gui/${uid_num}" "$plist" 2>/dev/null; then ok "$job: 未ロード → 自動で再ロードした" else ng "$job: 未ロード・再ロード失敗 (手動 launchctl bootstrap 要)" fi The string gui/${uid_num} is the target. id -u gets the logg