I fine-tuned a 1.7B model on data that doesn't exist - a 17-minute recipe

The full methodology behind localscrub's stage-2 specialist: synthetic training data with exact labels, training on the serving distribution, and why the fine-tune's first product is not accuracy - it's parseability. In the last article I benchmarked localscrub, a local-first PHI de-identification cascade, and buried a teaser near the end: a 1.7-billion-parameter model, LoRA-tuned in 17 minutes on a consumer laptop, went from producing zero parseable replies to 0.92 redaction recall on authentic clinical prose. This article is the recipe - every command, every hyperparameter, and the three methodology decisions that I think matter more than any of the numbers. The de-identification task is the case study, but the recipe generalizes to any structured-extraction job you want a small local model to do: if you can generate your training data and you train on the exact prompt your inference code sends, a specialist you can retrain from scratch in under half an hour is within reach on one 16 GB GPU. One constraint shaped everything: no real patient data anywhere in the loop. Not in the training set, not in the eval set, not in a single prompt sent to any model at any step. By the end you'll see that this constraint wasn't a handicap - the synthetic pipeline it forced turned out to be the most valuable asset in the project. Why fine-tune at all, when a pretrained classifier already wins? Fair question, because the last article's headline was an off-the-shelf 125M token classifier hitting 0.999 redaction recall. If that's available, why train anything? Because a token classifier cannot take stage 2's seat. localscrub's second stage has a conversational contract: it receives the note plus escalation hints - ambiguous spans that the rules engine flagged but couldn't resolve - and returns verbatim snippets with types, as JSON, adjudicating the escalations along the way. A token classifier tags a fixed label set; ask it about an identifier type it wasn't trained on and it has no opinion, and it can adjudicate "is this flagged span actually PHI?" only for the types it already knows. Only an instruction-following model can hold up the whole contract. A 14B generalist (qwen3:14b via Ollama) holds it up at 5-8 seconds per note. The question worth 17 minutes of GPU time: can a model an eighth that size be taught to? The training data costs nothing and leaks nothing The usual fine-tuning bottleneck is labeled data. Clinical de-id makes it worse: the gold-standard corpus (i2b2/n2c2 2014) sits behind a data use agreement, and real notes are radioactive - one mishandled training example and your privacy tool has a privacy incident. localscrub sidesteps both because its synthetic corpus generator already existed for evaluation. localscrub synth renders clinical notes from templates with fabricated identifiers planted at recorded character offsets - every phone from the reserved 555-01XX block, every domain from RFC 2606, every credit card Luhn-valid on a test prefix - deterministic from a seed. The eval harness scores against those exact gold spans. The insight that unlocks fine-tuning: a corpus with perfect gold spans is also a perfect SFT dataset. One command emits training pairs instead of eval records: localscrub sft -n 2000 --seed 7 -o data/corpora/sft-2000.jsonl Two thousand examples, generated in seconds, with two properties paid datasets can't match: - Zero annotation noise. The labels aren't human annotations of generated text - the generator planted the identifiers, so the gold is exact by construction. No inter-annotator disagreement, no boundary fuzziness, no label budget. - Zero exposure. There is no real PHI to leak because there is no real PHI. The training set is as regenerable and disposable as the eval set - corpora are gitignored, adapters are gitignored, and everything rebuilds from a seed. Seed discipline matters here: training uses seed 7, evaluation uses seed - Disjoint seeds mean no example overlap - but I'll flag the honest limitation now rather than in the fine print: the template corpus has only seven note skeletons, so train and eval share structure even though they share no content. Every template-corpus "after" number below is optimistic by construction. The MTSamples benchmark - authentic medical transcription prose that never entered the fine-tune in any form - is the honest generalization test, and it's the one I'll ask you to judge the recipe by. The one rule: train on the serving distribution Here is the decision I'd defend hardest, and the one I see skipped most often in fine-tuning write-ups: each training prompt is built by the actual inference code. localscrub sft doesn't format notes into some training-time template that resembles what inference does. For every example, it runs stage 1 on the note - the real rules engine, producing real escalation hints - and then calls localscrub.stage2.extraction_prompt , the same function the serving path calls, to build the prompt. The completion is the extractor's exact reply format, and it parses with the same salvage-tolerant parser inference uses. The tuned model never sees a prompt shape at inference time that it didn't see thousands of times in training, escalation hints included. Most "fine-tuned model underperforms in production" stories I've read trace back to exactly this seam: the training data was formatted by a script that approximated the serving prompt, and the approximation drifted. Eliminating the seam costs nothing - reuse the inference code - and it's free insurance. Two corollaries of the same principle: - Raw prompt→completion, no chat template. The serving path sends plain text, so training does too. This isn't just simplicity - format discipline is part of what the before/after numbers measure, and wrapping everything in a chat template would train a different behavior than the one being served and scored. - Completion-only loss. The model is graded on its reply, not on its ability to predict the note back. Standard practice, but it composes with the above: the loss covers exactly the tokens the serving path will consume. The recipe Hardware: one RTX 5080 Laptop GPU, 16 GB. Model: Qwen3-1.7B-Base (Apache-2.0). Total wall-clock for training: 17 minutes. # 1. training data: 2000 examples, seed disjoint from eval seeds uv run localscrub sft -n 2000 --seed 7 -o data/corpora/sft-2000.jsonl # 2. train (bf16 LoRA, ~17-30 min on 16 GB) uv run --group train python scripts/finetune_lora.py \ --data data/corpora/sft-2000.jsonl \ --out models/stage2-qwen3-1.7b-lora # 3. the authentic-prose benchmark (public MTSamples CSV, synthetic injections) uv run localscrub mtsamples --fetch -n 100 --seed 42 \ -o data/corpora/mtsamples-100.jsonl # 4. before/after as stage 2, standard eval harness uv run --group train python scripts/eval_lora.py \ --template-notes 20 --mts-notes 15 \ --mtsamples data/corpora/mtsamples-100.jsonl # before (base) - fewer # notes: ~30 s each uv run --group train python scripts/eval_lora.py \ --adapter models/stage2-qwen3-1.7b-lora \ --mtsamples data/corpora/mtsamples-100.jsonl # after (50 + 30 notes) Hyperparameters, all overridable via flags: LoRA r=16, α=32, dropout 0.05, all-linear targets; learning rate 1e-4 with cosine schedule and 3% warmup; 2 epochs; effective batch 16 (batch 2 × gradient accumulation 8); max length 2048; bf16; completion-only loss; seed 7. Nothing exotic - r=16 all-linear LoRA at lr 1e-4 is close to community defaults for this model size, and that's deliberate. The recipe's leverage is in the data (exact labels, serving distribution), not in hyperparameter heroics. If your first instinct on a weak result is to sweep learning rates, look at your data pipeline first. Deployment note: to serve the adapter through Ollama, convert it with llama.cpp's convert_lora_to_gguf.py and reference it from a Modelfile ADAPTER line. The evaluation script talks to the model via HF transformers directly, which skips that step for the before/after. Reading the before/after: parseability first, accuracy second The numbers, scored by the same harness as every other configuration in the project. Redaction recall is the safety metric - the fraction of gold spans whose every character is removed from the output; a half-redacted address counts as a miss, because a half-redacted address is still a leak. | config | corpus | unparseable replies | redaction recall | |---|---|---|---| | base 1.7B (before) | template, 20 notes | 20/20 | 0.66 (= stage 1 alone) | | base 1.7B (before) | MTSamples, 15 notes | 15/15 | 0.61 (= stage 1 alone) | | + LoRA (after) | template, 50 notes | 0/50 | 1.000* | | + LoRA (after) | MTSamples, 30 notes | 0/30 | 0.924 | * optimistic by construction - train and eval share note skeletons, as conceded above. The MTSamples row is the one to trust. Look at the "before" rows first, because they're the finding I didn't expect. The base model didn't merely underperform - it produced unparseable output on every single note, 35 of 35. Rambling, format drift, never once a reply the parser could consume. Since localscrub's merge is fail-closed, a useless stage 2 degrades the cascade to exactly the stage-1 baseline - the 0.66 and 0.61 aren't the small model helping a little, they're the cascade running as if stage 2 weren't there. After 17 minutes of LoRA: zero unparseable replies across 80 notes. The fine-tune's first product is not accuracy - it's parseability. That transition alone converts a dead stage 2 into a working one, before any detection improvements register. Accuracy followed: 0.61 → 0.92 redaction recall on prose the fine-tune never saw. If you're evaluating whether a small model can replace a big one in a structured pipeline, check format compliance before you check task skill - in my case it was the entire difference between "useless" and "viable," and it's the cheapest thing fine-tuning buys. What the remaining gap is made of - and how I know On MTSamples, the 17-minute specialist scores 0.924. The 14B