Your PDF-to-text pipeline is silently returning empty pages: detecting scanned PDFs and when OCR is worth it

If you feed PDFs into search, a RAG pipeline or an LLM, some of them come back with almost no text and nothing errors. Those are scanned PDFs: each page is a picture of text, with no text stored in the file.

This post shows how to detect them, what OCR settings matter, and what to expect from the result.

Disclosure: SpellzCode, the brand behind this account, sells a PDF-to-text tool (linked at the end); the code below uses free open-source libraries.

Detecting a scanned page

A normal PDF page stores its text. A scanned one stores an image. So: extract the text per page and look at how much came back.

// npm i unpdf
import { getDocumentProxy, extractText } from 'unpdf';
const buf = new Uint8Array(await (await fetch(url)).arrayBuffer());
const pdf = await getDocumentProxy(buf);
const { text: pages } = await extractText(pdf, { mergePages: false });
const empty = pages.map((t, i) => ({ page: i + 1, chars: t.replace(/\s/g, '').length }))
  .filter((p) => p.chars < 20);
const looksScanned = empty.length >= pages.length * 0.9;
console.log({ pages: pages.length, emptyPages: empty.map((p) => p.page), looksScanned });

Three cases come out of this:

• No empty pages: a normal PDF. Do not OCR it; the stored text is exact and OCR would only add errors.
• Every page empty: a scan. OCR it or reject it.
• Some pages empty: the common, annoying one. A typed contract with a scanned signature page, or a report with a scanned appendix. OCR only the empty pages.

Mark rows instead of guessing: a downstream system should know a document's text came from OCR.

OCR settings that actually matter

Using Tesseract (free, via tesseract.js in Node):

• Resolution. Render pages at 300 dpi (about 2,480 pixels wide for A4). In our tests at 150 dpi, digits in tables were misread regularly; at 300 dpi ordinary typefaces were read without error.
• Language. Set the document's language. With the English model, Swedish å, ä, ö or German ß come out wrong.
• Page segmentation. The default automatic mode (PSM 3) is right for normal pages. Tables come out one row per line, in reading order, not as a table.
• Confidence. Tesseract reports a confidence from 0 to 100 per page. Store it, and send low-confidence pages to a human.

// npm i tesseract.js unpdf @napi-rs/canvas
import { createWorker } from 'tesseract.js';
import { renderPageAsImage } from 'unpdf';
const worker = await createWorker('eng');
for (const { page } of empty) {
  const png = await renderPageAsImage(buf, page, {
    canvasImport: () => import('@napi-rs/canvas'),
    width: 2480,
  });
  const { data } = await worker.recognize(Buffer.from(png));
  pages[page - 1] = data.text;
  console.log(`page ${page}: confidence ${data.confidence}`);
}
await worker.terminate();

What to expect

• Clean printed text is read reliably.
• Check numbers. In one of our test pages, a typeface whose digits vary in height (old-style figures) had a 9 read as 0. If amounts matter, as on invoices, validate them (totals should add up).
• Handwriting, and phone photos of pages taken at an angle or in bad light, are not worth running through Tesseract.
• OCR is slow: seconds per page on a normal CPU. That is why OCRing every page "to be safe" is a bad default.

Using the tool instead

PDF Text Extractor & OCR on Apify does the detection and selective OCR above for a list of PDF links. It returns the text, optional per-page text marked as text or ocr, looksScanned, ocrPageCount, ocrConfidence, and the document's metadata:

// npm i apify-client
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('spellzcode/pdf-to-text-extractor').call({
  pdfUrls: ['https://arxiv.org/pdf/1706.03762'],
  includePages: true,
  ocrScanned: true,
  ocrLanguage: 'eng',
  maxOcrPages: 50,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0].pageCount, items[0].looksScanned, items[0].text.slice(0, 200));

It costs $5 per 1,000 PDFs, plus $8 per 1,000 pages that actually needed OCR. Broken links and password-protected files are free. Apify's free $5 monthly credit covers 1,000 ordinary PDFs or about 600 scanned pages.

Links

PDF Text Extractor & OCR
All SpellzCode Apify actors