Noisy Text in RAG: Typos, OCR, and the Gap Classical Spell-Check Leaves

Noisy Text in RAG: Typos, OCR, and the Gap Classical Spell-Check Leaves Enterprise Document Intelligence [Vol.1 #B1] - Three sources of one problem. User typos, fast-typing transcription noise, OCR character errors. Classical spell-check handles one of them. Embeddings carry the rest The user types “assurance décénale” and the document says “décennale.” One missing letter, and a literal search finds nothing. Real questions arrive with typos, and real documents have their own; before retrieval can match anything, someone has to fix the spelling on both sides. This article is a bonus in Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks. It tackles noisy text across the pipeline: user typos, fast-typing transcription noise, OCR character errors, what classical spell-check fixes, and what embeddings have to carry. 🧭 New to the series? Every article in this series sits on our two Towards Data Science author pages, Angela Shi and Kezhan Shi. That is the shortest way to see what is covered and where this one sits. 📓 Runnable companion notebooks are on GitHub: doc-intel/notebooks-vol1. The same problem shows up on both sides of a pipeline. On the question side, type “wat is teh covarge for fyre damge?” into a chatbot built over a company knowledge base: three typos and a missing letter, and the chatbot returns nothing useful until the question is retyped carefully. On the document side, dump 50,000 customer support tickets into the same pipeline for retrieval: half of them are written in fragments, abbreviations, mixed case, with the same kind of errors, and the pipeline that worked for clean queries against clean documents starts returning noise. This is the noisy-text problem in enterprise RAG. It looks like a spell-check problem from the outside, but the actual cause splits three ways. The user mistyped a word (a typo). The user typed under pressure on mobile and scrambled boundaries, dropped accents, abbreviated (transcription noise). The document came through OCR and a scanner silently replaced O with 0 , broke a fi ligature, split policyholder into policy holder (OCR noise). All three end with the same symptom downstream: a token in the query or in the document does not literally match what it should, even though the meaning is intact. The classical spell-correction toolbox was built for one of the three. The other two are the ones that hurt enterprise pipelines, and the ones embeddings are quietly built to absorb. 1. Forty years of classical spell-correction Before embeddings and LLMs, spell-correction was a solved engineering problem. Five techniques cover most of what ran in production between 1980 and today, all with mature Python libraries (rapidfuzz , jellyfish , symspellpy , pybktree ). The next subsections walk through each, then close with the case where this toolbox solves the problem. 1.1 Levenshtein distance The minimum number of single-character edits (insert, delete, substitute) needed to turn one word into another. The foundation under almost every spell-checker built in the last forty years. A misspelled word’s “best correction” is the dictionary entry with the smallest Levenshtein distance, broken by frequency in case of ties. The full algorithm runs in O(n·m) time: fast on a single word, but on a 50,000-word document it means one comparison matrix per word pair, which adds up quickly. 1.2 BK-tree A Levenshtein query against a million-word dictionary is too slow if you compute distance to every entry. The Burkhard-Keller tree (1973) indexes the dictionary so that all words within distance k of a query are reachable in roughly O(log n). This makes aspell and hunspell feel instant. No machine learning, no GPU, just a clever index built on triangle inequality. 1.3 Soundex and Metaphone Phonetic codes. They map words that sound alike to the same key regardless of spelling. Designed in the 1910s for U.S. census name matching, still useful today for surname lookup, drug-name disambiguation, voice-to-text post-processing. Run on six near-homophone pairs, the two coders mostly agree but disagree on the harder spellings, which is why production systems carry both keys: Phonetic matching catches the kind of variation that Levenshtein misses: a French speaker writing Stéphane as Stefan , an English speaker writing Catherine as Kathryn . The price is that any two unrelated words that happen to sound alike collide. 1.4 SymSpell The modern fast variant. Precomputes all deletes within distance k for every dictionary word and stores them in a hash. Lookup becomes a hash join, sub-millisecond on a 100k-word dictionary on a single CPU core. Frequency breaks ties. The dictionary is best built from the target corpus itself, not a generic word list, so corrections land on terms that appear in the documents the user is searching. 1.5 Character n-grams Index every word as a set of overlapping n-character substrings, then score similarity by Jaccard overlap (the fraction of substrings two words share: eight matching trigrams out of nine gives 0.89) on those sets. Catches near-matches even when the misspelled word is not in the dictionary. The basis of every modern fuzzy-search index that does not rely on a curated dictionary (Elasticsearch ’s edge-ngram analyzer, pg_trgm in PostgreSQL). 1.6 Where this all works Hand the toolbox a single misspelled word with a clear correction in the dictionary, and it solves the problem every time, in microseconds. Take a typical RAG query with a single typo: The computed distances confirm the ordering: coverage lands at distance 1 alone, every other valid candidate sits at 2 or more. coverage wins by a clear margin, fast and deterministic, with no GPU. For this shape of problem, classical methods are still the right tool. The trouble starts when the shape of the input no longer fits this assumption. 2. Where the classical playbook breaks Classical spell-correction was built around three assumptions: the user typed one word at a time, the typo produced a non-word, and the dictionary was the ground truth. Real enterprise queries violate all three. 2.1 The typo produced a valid word This is the biggest hole in the classical toolbox. When a typo lands on another correctly-spelled word, no spell-checker flags it as an error: there is nothing to flag, both spellings are valid. The mistake is not in the orthography but in the fit between word and context. The six pairs side by side with their distances and meanings make the trap visible: A user types “what is the overage on my homeowner policy?” in an insurance chatbot. They almost certainly meant coverage (the amount the policy will pay out), not overage (the excess amount they owe past a limit). A classical spell-checker has nothing to flag: overage is in the dictionary, the SymSpell lookup returns it as the best-confidence match for itself, and the retrieval layer happily fetches documents about paying overages on usage caps, not coverage limits. The user gets a confident wrong answer. The reason classical methods cannot catch this is structural. They score similarity against the dictionary. Whether the word fits the query’s domain is a different question entirely. Answering it requires reading the surrounding text and knowing that “homeowner policy” and “coverage” co-occur in the corpus far more often than “homeowner policy” and “overage” do. That is what embeddings encode (Article 2). It is also what no Levenshtein-style method has any way to access. 2.2 Word boundaries are wrong, not the letters The other assumption the classical toolbox makes is that the input is a sequence of well-separated words. When users type fast (especially on mobile, especially under stress), they scramble word boundaries. They write policy holder as policyholder , or non-employee labor as nonemployeelabor , or split homeowner into home owner . Sometimes they merge two questions into one fragment with no punctuation. OCR adds the same problem from the document side. A scanned PDF passed through Tesseract or AWS Textract returns text with broken word boundaries on tight kerning, missed accents, and stray punctuation. A 1% character error rate on a 500-page PDF is 25,000 broken tokens. Many of those broken tokens are valid words after the boundary error: policyholder becomes policy holder becomes the trigram set of two unrelated words. The unit a classical spell-checker is built to fix is one word at a time. The unit that breaks in fast typing or noisy OCR is a sequence of terms. As soon as the boundaries are unreliable, Levenshtein has nothing to anchor to. This is the gap classical spell-correction never closed. 2.3 OCR replaces letters with other letters Boundary scrambling is the loud OCR failure, but the silent one is worse. Modern OCR engines confuse certain glyph pairs in ways the human eye barely notices, and the result is text that looks almost right but does not match anything a literal search would look for. The original character is gone, replaced by a near-identical one. No misspelling rule flags it, because nothing was misspelled. The character was misread. Now stack this against grep , the literal-search baseline every enterprise has on top of its file shares: A distance of 4 on a 27-character phrase is the borderline the classical playbook cannot survive. Raise the fuzzy-search threshold to 4 and false positives explode (any unrelated 27-character phrase at distance 4 matches too). Drop it to 2 and the genuine OCR’d form is missed. There is no setting of the threshold that holds both ends. The asymmetry is the point. OCR distributes noise per character. Search terms in enterprise queries are per phrase. The two scale differently, and Levenshtein has nothing to bridge the gap. The next section shows what cosine similarity does with the same corrupted phrases. 3. Embeddings and LLMs handle this naturally, but for different reasons Article 2 of the series showed that embeddings (dense numerical represe