Cloud bills kept climbing from 24/7 AI development - I moved the decisions and the implementation to my own local LLMs and cut the cost

From late June into July 2026, I rebuilt the execution backbone of my development onto my own local large language models (LLMs). The trigger was cost. The more you hand development over to AI, the more every task-routing decision and every actual code generation gets sent to a cloud AI, each and every time. Run it 24 hours a day without stopping, and that usage-based billing piles up every month, exactly in proportion to how much you used. And this is something you keep paying, month after month, forever. But, I thought. If I buy the hardware once and shift the execution beyond that point onto my own local LLMs, couldn't I erase most of this ongoing cost? Couldn't I replace usage-based billing - which grows with every bit you use - with a one-time hardware cost? Here is what I did. I moved the task-routing decisions (which I'll call the "orchestrator" in this article) and much of the hands-on work from the cloud AI to my own local LLMs. To do that, I bought one NVIDIA DGX Spark and combined it with the four Macs I already had to build an execution backbone that development tasks flow through. Leave the hands-on work to the AI, and keep only the decisions for the human. Push that division far enough and you arrive at Human-Out-Of-The-Loop (HOOTL), where the human steps outside the loop. Let me say this up front. This is not a bragging-rights story about "I cut ¥X." What I want to convey here is not a proof of some dollar amount, but the way you wire things so you erase as much of the recurring cost - the usage-based billing - as the design lets you erase. Let me put the conclusion first (reading the body takes about 8 minutes). - Erased as much of the recurring billing as the design allowed - took both the decisions and the hands-on work locally first, and sent to the cloud only what genuinely needed it - Locked the buy/no-buy criteria with an ADR (a record of design decisions) before purchasing - ruled out "buy it because I want it," and decided the purchase on measured speed alone - Decided which model goes where, per role, from measured speed and cost - routing decisions on 14B, hands-on code generation on the zero-billing 72B lane - Ran one model per host - avoided swap costs and made the split between local and cloud visible in numbers When you actually do this replacement, the things to decide narrow down to three. ① Whether to buy your own hardware, and if so, which. ② Which model to put on which job. ③ How to line up multiple machines. I'll write them in order. The period in question breaks down like this. The purchase decision itself was an immediate call from the benchmark results alone (late June 2026). From there the DGX Spark arrived (July 10), and it took about two weeks to get the basic form working. After that, in late July, an incident occurs. From the purchase decision to here is about one month of records. Before buying: to erase the recurring billing, I first decided "whether to buy my own hardware" by measurement Why did I want to shift the decisions onto my own local LLMs in the first place? There are two reasons. One is the recurring cost. The other is that I don't want the decision-making itself to depend on external billing. The routing decision is the central process that runs 24 hours a day. Leave this to the cloud, and the cost of the central process piles up every month in proportion to use, and if it stops for reasons on the external service's side, all of development stops with it. So I put no cloud LLM on the orchestrator. This is a design principle of local-commander (that's the name of this development orchestrator). Cloud costs stop development in two forms. On a free tier you hit the usage limit and stop; on usage-based billing it piles up without limit, in proportion to use. Neither is a fantasy for us. Our organization's GitHub Actions stopped from April 2026 on suspicion of hitting the free-tier ceiling. I ran the numbers, too. Running CI on GitHub Actions' standard runners (Linux, 2 cores, $0.008/min) for two machines' worth, even estimating utilization at half (12 hours a day), comes to about ¥55,000/month. Against that, a CI server is ¥130,000 each, ¥260,000 for two, one time. It pays for itself in about five months, and beyond that there is no recurring billing. That said, shifting the decisions local requires speed. Running a 31B-class model on the MacBook Pro (M3) at hand gives an effective 5 tok/s, 60-200 seconds per decision. That doesn't reach the speed needed for 24-hour autonomous operation. Here, so that "I want a DGX Spark" as a want-it impulse wouldn't come first, I bound myself before buying with an ADR (Architecture Decision Record - a document that records design decisions). ADR-0003: Gate the DGX purchase decision on PoC (proof-of-concept) benchmark results only On top of that, I ran a 16-case classification benchmark. The results are these. - The system's final classification matched 16 of 16 (100%), with 0 dangerous-side misclassifications (qwen3.6-35b) - But this 100% does not mean "the LLM alone is perfect." The raw LLM's JSON-formatting success was 69% (31% fallback), and broken output was absorbed by deterministic safety overrides on the code side, lifting the final decision to 100%. The 100% is not the model's cleverness but the result of a mechanism that fails toward the safe side even when it breaks - The only bottleneck is speed (60-200 seconds per decision on M3) The correctness and safety of the final decision can be guaranteed by safety rules on the code side even when the LLM breaks. What's missing is only speed, and speed can be solved with hardware. The day after this measurement lined up, I decided to buy. This connects to the recurring-cost story. Hardware is done once you buy it. The execution beyond that carries no per-token billing. Replace the monthly usage-based billing with a one-time expense. I positioned it as a purchase for exactly that. The PoC report and ADR I used for the decision are all kept in the repository. The day it arrived: the first thing I did after ssh 2026-07-10, the DGX Spark went live. This is the first session where I ssh dgx 'd in. aarch64 on GNU/Linux 6.17, 3.67TB of storage. From the day it arrived, I'd already started running model-selection benchmarks The terminal and dashboard images in this article have only the text - product names, hostnames, IPs, and so on - replaced with anonymous dummies from the actual screens (layout and numbers are real data; only verified numbers are shown). Some screens are anonymized diagrams with the layout reconstructed. I bought this machine intending to place it at the center of the execution backbone that replaces the monthly usage-based billing. So from day one, I began setup on the premise of running local-commander (the first thing I placed was a marker called .lc-bootstrap-allow that permits its initial startup). Selection: "bigger models are smarter" did not hold for classification The DGX has 121GiB of memory. It loads a 72B class with room to spare. Naturally the urge comes up: "I want the biggest one as the orchestrator." Here too I chose by measurement. Here is the result of comparing 14B / 32B / 72B on the same cases for the orchestrator's job (task classification). | Model | Classification accuracy | Time per decision | |---|---|---| | qwen2.5-coder:14b | 100% | 4.4 s | | qwen2.5-coder:32b | 88% | (intermediate) | | qwen2.5:72b | 100% | 22.8 s | 14B and 72B tie on accuracy, and 72B is 5× slower. And only the intermediate 32B drops to 88% from broken JSON formatting - a counterintuitive result. For a task like classification, "bigger is smarter" does not hold. So I decided on 14B for the orchestrator. So is 72B wasted? No. I turned 72B into the "zero-billing work lane" that runs the hands-on code generation. This is where the recurring-cost story bites. Throwing the hands-on work at the cloud incurs usage-based billing, but running it on the 72B already loaded on the DGX incurs no per-token billing. Just to be safe, though - I put 72B on the hands-on work not because "72B is smarter at code generation." In the generation benchmark, 14B and 72B tie at a 57% success rate, and 72B is about 6× slower, and even the primary source concludes "promotion to 72B is shelved." I still wired up the 72B lane because I can reuse an asset already loaded on the hardware at zero per-token billing. The reason I put 72B on the work even though 14B is faster is that the four 14B Macs are always full with classification and have no spare, while the DGX has 72B loaded from the start (how these hosts are split is the next section). I limit its use to low-risk tasks classified as local-ok (judged able to complete safely on local), and if it fails, escalate to the cloud (Claude). On that premise, 72B's slowness is acceptable, and I gain one more lane that carries no billing. Configuration: the DGX is a 72B-only machine, the Macs are 14B classifiers The final configuration is one model, one host. - DGX Spark: a qwen2.5:72b (111.4GB) dedicated machine. As a "work lane" that carries no per-token billing, it processes low-risk ( local-ok ) code-implementation tasks 6 in parallel, and escalates to the cloud on failure - Four Macs (M4 / M3 and others): a pool of qwen2.5-coder:14b classifiers, bundled with a weighted load balancer (a mechanism that distributes load by weight). Apple Silicon, with its GPU and unified memory, is inherently suited to LLM inference. On top of that they double as CI runners, saving the pricey cloud-build (macOS runner) billing for iPhone apps. They pull many roles, so if I was going to buy, they were the optimal choice - Cloud (Claude / Codex): only tasks that don't complete locally flow through, passing a human approval gate - CI server: separately from the above, I've also introduced a CI server to keep the orchestrator (local-commander) from stopping (the cost estimate is above). Availability (HA) and configuration details I'll write in a separate article The a