iCloud Silently Evicted 69 Article Files and Killed 4 Days of Publishing: EDEADLK and a read_text_resilient Design

Every one of my publishing lanes went dark for four days, and every script involved exited with status 0. Nothing had crashed. The files themselves had quietly stopped existing on disk - macOS had uploaded them to iCloud and deleted the local copies to "optimize storage." Why This Matters What it means for automation to depend on its environment When you run 160+ launchd jobs around the clock, the execution environment itself becomes a failure source before your script logic does. Ports get exhausted, processes orphan and pile up, memory never frees - I wrote about that class of resource leak last time. This is a completely different kind of total failure that happened the very next day. The files had become fatal to read. Not a bug in my code. Not a filesystem bug. An unintended side effect of a mechanism macOS runs under the name "optimization." What optimize-storage actually does macOS's "Optimize Storage" (System Settings → General → Storage → Optimize Storage), on a machine with iCloud Drive enabled, uploads files under Desktop and Documents to iCloud and deletes the local copies when free disk space gets tight. In Finder they still look like normal icons, but there is no local data - they are in a "dataless" state. Click one and it downloads automatically. For a human user, that's an acceptable tradeoff. The problem is automation scripts. python3 's open() , pathlib.Path.read_text() , cat , jq , cp - all of them die instantly on a dataless file with Errno 11: EDEADLK: Resource deadlock avoided . The name "Resource deadlock" makes you suspect a deadlock, but this is a POSIX errno code that macOS repurposes to mean "waiting for a file download." No lock is contended. No thread is stuck. The mere fact that "the data isn't local" surfaces to the process as a fatal error code. You can also get EAGAIN (resource temporarily unavailable). That one shows up as a race right after a download starts. The actual damage: four days of zero posts On August 6, 2026, note's automated publishing stopped across every lane. The error log was a wall of EDEADLK. Digging in, 69 article files under ~/Desktop/Article/ had gone dataless. Disk usage had hit 98% (25GB free), and iCloud had silently evicted the data at that moment. That's exactly where the note lane was reading from. It couldn't read, so it couldn't post. It couldn't post, so there was no revenue. Starting August 3, four days of zero posts. The lane producing most of my revenue had stopped in a form that showed neither errors nor "failure" alerts at a glance - that's what stung most. The script returns exit 0 . It's treated as "no articles found, so nothing to do," so the liveness check passes as "success." Silent failures are the ones you discover last. The previous day's (August 5) incident was "resource leak: memory and orphaned processes." A different kind of total failure arrived one day later, and both present as "all lanes stopped." The root causes differ; the surface is identical. Without a triage pattern, you burn an hour every time. The Vault had the same problem Something else I noticed the same day while working: the Markdown files under my Obsidian Vault (~/Documents/claude-obsidian/ ) were repeatedly going dataless too. CLAUDE.md and my learning notes get re-evicted as long as disk pressure continues, no matter how many times I materialize them with brctl download . This isn't a Desktop-only problem. Every folder synced to iCloud Drive is in scope. Documents included. As long as your automation's read/write targets live there, the same thing happens every time the disk fills up. The Overall Picture The permanent fix has two pillars. 【問題の構造】 ディスク 98% → optimize-storage 発動 ↓ Desktop/Documents 配下のファイル → iCloud退避（dataless） ↓ 自動化スクリプトが open() → EDEADLK → 即死 → exit 0（静かな失敗） ↓ note レーン 4日間ゼロ投稿（8/3〜8/6） 【恒久策 A: ファイルを逃がす】 ~/Desktop/Article/ → ~/content/article/ （実体を移動） ↓ ~/Desktop/Article → ~/content/article のsymlink残置 （移行中もジョブを壊さない） ↓ パス定数を JS/MJS 6本 + Shell 3本 = 9ファイル更新（commit a1b37dc） 【恒久策 B: 読む側で EDEADLK を吸収する】 Documents/Desktop 配下を読む箇所 ↓ read_text_resilient(path, attempts=4) を挟む ↓ EDEADLK / EAGAIN のときだけ → brctl download （実体化をカーネルに要求） → 2秒 → 4秒 → 8秒 の指数バックオフでリトライ それ以外の OSError → 即 raise（握りつぶさない） 4回目も失敗 → OSError をそのまま raise Fix A: Move automation's read/write targets out of Desktop/Documents This cuts the problem off at the root. I moved the actual data of 796 article files from ~/Desktop/Article/ to ~/content/article/ and left a symlink at the original path. Even if a launchd job fires mid-migration, it can still read through the symlink, so the job doesn't break. Updating the path constants is a one-line replacement per script. This time I just replaced Desktop/Article with content/article across 9 files total (JavaScript, MJS, Shell) and confirmed zero hits with a residual check (grep -r 'Desktop/Article' ) - commit a1b37dc . This is the main line of defense. If you exclude ~/content/ from iCloud Drive sync, or simply put it on a non-iCloud-managed path directly under ~/ , no dataless eviction happens even under disk pressure. Don't put automation's read/write targets in Desktop or Documents - that's the structural fix. That said, for places where automation reads from a folder that needs iCloud sync - like the Vault (~/Documents/claude-obsidian/ ) - moving files doesn't solve it. That's where Fix B comes in. Fix B: Absorb EDEADLK with read_text_resilient Wrap code that reads from Documents or Desktop in a function with retry tolerance for EDEADLK. import errno import subprocess import time from pathlib import Path def read_text_resilient(path: Path, attempts: int = 4) -> str: """ iCloud の dataless ファイルに対して brctl download → 指数バックオフでリトライ。 EDEADLK / EAGAIN 以外の OSError はリトライせず即 raise する。 """ delays = [2, 4, 8] # リトライ間隔（秒）: 初回は sleep なし last_err: OSError | None = None for i in range(attempts): try: return path.read_text(encoding="utf-8") except OSError as e: if e.errno not in (errno.EDEADLK, errno.EAGAIN): raise # 権限エラー・存在しないパスなどは即死させる last_err = e if i is a macOS command that asks the kernel to re-download a file evicted to iCloud back to local storage. Sync takes time, so the waits grow 2s → 4s → 8s. If the fourth attempt (attempts=4 ) also fails, it throws that OSError as-is. I deliberately do not build a fallback that swallows the exception and returns an empty string. Returning empty string gets interpreted as "the file was empty," processing continues, and you reproduce exactly the silent failure of "the data vanished but it counted as success." I added this function to obsidian-notion-sync/sync.py and applied it at the 4 places that read .md files under the Vault. All 51 existing tests pass. Why a "silent fallback" is poison Let's pause here. An implementation that swallows EDEADLK and returns "" is easy to write. try/except into an empty string and "no error occurs." But what happens if you do that? The script that reads articles and assembles posts decides "I read an empty file," concludes "no articles to post today," and returns exit 0. No alert fires. The liveness check passes as "healthy." You don't notice until you open the note dashboard - exactly as I failed to notice for four days. The smarter the fallback, the later you find the problem. In automation, a "silent failure" is far worse than a loud one. A design that hates noise so much it erases the signal too makes the post-incident cost of failures skyrocket. EDEADLK is an unambiguous state - "the data isn't local" - so retrying is meaningful. But if you retry four times and all four fail, then either "the environment is broken" or "the file itself is the problem," and the correct move is to propagate it upward as an exception. What the caller does about it is the caller's decision - that's how you place responsibility in error handling. A triage pattern: separating three flavors of "everything is down" In the actual incident response, multiple causes overlapped on the same day. - Resource leak (memory, orphaned processes): the main cause on 8/5, the day before - EDEADLK from iCloud dataless files: the main cause on 8/6 (the subject of this article) - Weekly quota exceeded ( You've hit your weekly limit · resets 10am ): running in parallel the same day All three look like the same surface symptom: "every lane is stopped." Dive into "everything is down" without a triage pattern and you'll burn time on a different cause while investigating memory. The triage order is as follows. ① Look at the error code first. EDEADLK means a file problem. Timeout means a process/memory problem. You've hit your weekly limit means quota. Error messages don't lie. ② Look at what is not dying at the same time. In this case: if it's an iCloud problem, Python's open() dies but curl doesn't. If it's a memory problem, both Chrome and claude -p die. The breadth of the failure narrows down the nature of the cause. ③ Check free disk space. The root cause here was "disk at 98%." One command, df -h ~ , shows it. If you're not under pressure, iCloud dataless eviction doesn't happen. In the next section, I'll get concrete about the pitfalls I hit actually applying read_text_resilient , and what I screwed up during the migration off Desktop. Implementation Details Design decision ①: Why narrow by errno The core of read_text_resilient is that instead of retrying every except OSError as e , it retries only when e.errno is EDEADLK or EAGAIN . Why not retry everything? OSError subclasses include a huge number of errors that will never succeed on retry: PermissionError (errno 13), FileNotFoundError (errno 2), IsADirectoryError (errno 21), and more. Trying to read a file you lack permission for four times just returns PermissionError four times. brctl download has no effect whatsoever on a permission error. Only EDEADLK (errno 11) and EAGAIN (errno 11/35) indicate a state of "the data isn't local right now, but downloading it might make it readable." EDEADLK is when iCloud doesn't hold the data locally; EAGAIN is a race condition