The same input gave me a different translation every time. The bug wasn't where I thought.

I kept re-running the exact same input through my translation app. Same code. Same model. Same everything. And the word "machines" kept flipping between two different translations. Sometimes it came out as "機械" (machine). Sometimes as "あなたのPC" (your PC). No code changed between runs. No input changed either.

My first assumption was a race condition somewhere in my pipeline. It wasn't.

Where I actually looked

I checked the obvious suspects first: caching, threading, anything stateful that could make the same input behave differently on different runs. All clean.

So I went one level deeper, into how the model picks the winning word. Translation models score every candidate word and pick whichever scores highest. When I logged the actual scores for "machine" vs "your PC" on this input, they were almost exactly tied. That's the part that mattered.

When two candidates are separated by a tiny margin, the order floating-point operations get summed in can nudge the score just enough to flip which one wins. Same math, same inputs, different accumulation order between runs - and a near-tie flips sides. Nothing was actually random. It was deterministic all the way down. It just wasn't deterministic in a way I could predict, because the thing that decided the winner was rounding noise several layers below anything I was testing.

The fix wasn't "make it deterministic"

Forcing strict floating-point determinism across an ML pipeline is its own rabbit hole, and not one I wanted to go down for one word. Instead, I looked at why the tie was so close in the first place. "Machine" and "your PC" were close enough in meaning, in this context, that the model wasn't confident either way.

So I widened the margin instead of trying to eliminate the noise: I swapped the input word choice from "machines" to "equipment," which the model was much more decisively confident about. Scores stopped being close enough for rounding noise to matter. The flip-flopping stopped.

A trap I almost fell into

I want to be honest about a trap I almost fell into here: my first instinct was to just blanket-replace every instance of "machines" with "equipment" everywhere. That would've been wrong - in plenty of other sentences, "machines" was already translating correctly and consistently. A global find-and-replace would've quietly broken correct translations to fix one flaky one. I only changed it in the specific context where the tie actually existed.

What I'd tell someone hitting this

If the exact same input gives you different output on different runs, and you've ruled out caching, threading, and anything stateful - look for close scoring ties, not randomness. "Nothing changed" and "nothing is random" can both be true at the same time; the instability can live in rounding order underneath a scoring step you're not even looking at.

And once you find a fix, don't trust it from a single passing run. I re-ran the same input across multiple sessions before I believed the fix actually held, since the whole bug was runs disagreeing with each other in the first place - one clean run doesn't prove anything if the failure mode is "some runs differ."

I write more about building this translation app (and the bugs it keeps teaching me) on Threads.