A therapy scribe where a model may add the commas, but code decides whether it may change a word

The evening notes problem

A therapist who sees clients back to back writes the notes afterwards, often in the evening, from memory. ClinicLog gives that evening back.

With the client's consent, the therapist records the session on a phone. An hour in Hebrew, English or both becomes a transcript, then a draft note in the therapist's own template. The therapist reads both, fixes what is wrong, and signs the note.

A speech recognizer plus a model told to tidy up fails twice here. Israelis switch language mid-sentence, and on the first day a recognizer set to Hebrew turned the English half into nonsense. And a tidied transcript is a falsified record: told not to tidy, a model still turned "not especially" into "especially".

Figure: Example. The lines and the model's edits are invented; the verdicts are what ClinicLog's checks return for these exact lines. The English under each line translates it. See the animated figure in the original note.

Code, not the prompt, decides

The calls I made: Code, not the prompt, decides what the model may change. A stricter prompt was the easy answer, and I turned it down: the prompt already said not to tidy, and the model dropped the "not" anyway.

So the model writes, and plain code with no model in it checks every line it returns. The punctuation pass may add marks but not change a word. The repair pass may fix a misheard word, but not drop a word on its negation list. A line that fails goes back to the words that were heard, on its own; the other lines keep their edits. A repair that passes stays marked, so the therapist can see what was heard.

I measured on real voices, and went against my own test data. On that first day I switched to a multilingual recognizer, because a Hebrew-only one cannot hear the English half. My synthetic Hebrew clips then said Hebrew-speaking practices would do better on the Hebrew-only model. Public recordings of real voices said the opposite: it was worse on three of four Hebrew clips and tied on the fourth. For English, the single-language model did cut word errors by about a third. [1]

So a practice that declares English gets the English model, and one that declares Hebrew keeps the multilingual one. Trusting the synthetic answer would have sent Hebrew practices to the worse recognizer.

More context, fewer guesses

The repair model works out what a garbled word meant from the lines around it. Given 20 turns at a time, it proposed seven repairs on the public test recordings and turned "compassmented" into "compartmentalized". Given 60, it proposed two and wrote "compos mentis", which is what was said. Less context made it guess more, not less, so repairs travel 60 turns at a time. [2]

What happened

Recording the session itself has been on in Israel since 24 August 2026, and stays off in the US; dictated notes came first, in November 2025. [3] I am ClinicLog's only engineer.

Synthetic evals of the note and the speaker roles, on short invented exchanges in Hebrew and English, pass 14 of 14. [4] Those are synthetic and say nothing about clinical accuracy. That is what the therapist's review and signature are for.

Under the hood

A language model makes four passes over a session, and code decides what each may produce: punctuation, repairs, speaker roles, and the note.

• In the room: The phone records the session and uploads it in pieces while it runs.
• Speech service: The voices are separated and Hebrew and English are transcribed in the same sentence. Our copy of the audio is deleted once the service accepts it.
• Model, then code: A model adds punctuation. Code compares the words before and after, and a turn that changed goes back to what was heard.
• Model, then code: A second pass may fix a misheard word. A change that drops a listed negation is refused; every change that passes is marked.
• Who is who: A model reads the opening of the session and says which voice is the therapist. It answers with roles, never names.
• The draft: The note is written from the text, into each section of the therapist's own template, in one call.
• The therapist: They fix lines and voices, edit the draft, and approve. The note stays a draft until they finalize it.

Record - uploaded as it runs / Recognize - voices separated · he + en / ↑ audio ↓ text only / Punctuate - in code every turn checked / Repair - in code every repair marked / Speaker roles - therapist · client / Draft note - one field per section / Review - the therapist signs

How the pipeline survives a lost worker is its own note. This one is about the words.

The punctuation check

Both versions of a turn are split into words, everything Unicode classes as punctuation or spacing is removed, and the two word lists must match. A space inside a word fails; a new paragraph does not.

The check, and its caller:

def is_insertion_only(original: str, revised: str) -> bool:
    """True when `revised` differs from `original` only in punctuation/spacing. One character of difference in the compared stream returns False, which the caller must treat as a rejection and fall back to the raw text."""
    return fidelity_tokens(original) == fidelity_tokens(revised)

for text, candidate in zip(batch, candidates, strict=True):
    if is_insertion_only(text, candidate):
        refined.append(candidate)
    else:
        logger.warning("verbatim_readability_rejected")
        refined.append(text)
        rejections += 1

The first version ignored invisible formatting characters. A review before the pass went live wrapped a number in a Hebrew sentence in a direction override: the check passed it, and a browser shows the digits reversed. Models add direction marks of their own unasked, so invisible characters are compared now.

And on the first real recording the model deleted the dashes that mark a measured pause; to Unicode a dash is punctuation. Pauses are now compared like words.

The repair guards

Negations, from a Hebrew and English list, are counted before and after and must match. A repair may change at most half of a turn's words, or two in a short one, and never all of them. The changes are computed from the text, never self-reported, and each one reaches the therapist and the note model with the word that was heard. [2]

Language, roles and the template

On go-live day a synthetic mixed clip scored 2.4% word errors on the multilingual model against 26.8% Hebrew-only, and I switched that evening. [3] Automatic language detection is no answer: it picks one language per file. On a two-person interview from the public set, multilingual scored 5.7% against 8.7% Hebrew-only. [1]

The speech service separates voices without knowing whose they are. A model reads the first 40 turns, each cut to 240 characters, and may answer therapist, client or other, never a name. [5] Two therapists or two clients is hedging, and the answer is dropped. The therapist flips a wrong voice with a tap.

The note is written from the transcript, never the audio. Audio costs more than text, and when I gave the repair pass the audio too, it stopped repairing and started transcribing again: word errors on a public clip went up, not down.

Until 21 August a structured template got the whole note in its first section. Now the response schema is built at run time with one property per section, described by the template's own guidance for that section, so the plan lands in the plan section in the same call.

The roles failure that looked like a cautious model

Speaker roles shipped on 24 August. The next day the worker log showed they had never run. I had declared the answer as a mapping of voice to role, and the model provider's library refuses that schema shape before sending anything. My code caught the error, as designed, returned nothing, and the screen showed "Voice 1" and "Voice 2": exactly what a model that could not tell looks like. Every test faked the call. The answer is now a list of pairs, and a test checks the schema for the shape the provider refuses.

The working call then exposed two holes the broken one had hidden: the recognizer's bucket for unattributed words could be labeled client, and the hedging check caught two therapists but not two clients. Both were closed that night. [5]

What transfers

• Check the model's output in code, item by item. A prompt is a request; a check runs every time, and a line that fails reverts alone.
• Measure on real voices before trusting synthetic ones. The synthetic clips would have sent Hebrew practices to the worse recognizer.
• A fallback that degrades to nothing needs a test of its own. Otherwise the failure looks like a model being careful.

Sources

• Measured 30 Aug 2026 on public recordings: English 9-23% word errors fell to 6-17% English-only; Hebrew-only was worse on 3 of 4 Hebrew clips, 5.7% to 8.7% at most
• 102 tests on the punctuation check, repairs, speaker roles and template split, passing 1 Nov 2026.
• Repair batch, 30 Aug: 7 edits with 20 turns of context, 2 with 60
• Dictated notes since 9 Nov 2025; session recording on in Israel since 24 Aug 2026, off in the US. That day a mixed clip: 2.4% word errors multilingual, 26.8% Hebrew-only
• Synthetic evals, re-run 1 Nov 2026: 7 invented exchanges of 3 to 5 turns, Hebrew and English, each with name masking off and on: 14 of 14 passed, 1.2 to 3.5 s a call
• Speaker roles read the first 40 turns, 240 characters each. Shipped 24 Aug 2026, never ran on two voices until the next evening's fix; a test now checks the schema
• Originally published at yussieik.com, where the figures are animated.