OCR couldn't tell 720p lecture text from 1080p, a cropped SSIM could

When I re-encode a screen recording, all I really want to know is whether the small text survived. "Looks fine to me" is not a measurement, and a full-frame score mostly grades the flat background.

So on one test recording I measured the text the narrow way: same moment from every encode, same rectangle of small text, scored against the original with SSIM. I also ran OCR on that strip, expecting it to be the stricter test. It was the weaker one: three 720p encodes got 1 or 2 OCR errors, the same as the untouched source, while SSIM fell from about 0.97 to 0.88.

The sample is synthetic. It is a homemade 4-minute fake lesson with four slides, a 15 px handout column, a code editor being typed into and a moving pointer, rendered frame by frame from a web page rather than captured by a screen recorder. It was encoded to H.264 at 1920×1080, 30 fps and a constant 12 Mbps, a bitrate I chose rather than any recorder's default: 360.0 MB in decimal megabytes, no audio.

About 78% of it is a still screen with no sensor noise, so a real screen capture would be harder to squeeze. Every number is a single run on an Apple M4 Mac with ffmpeg, HandBrake's command-line build and macOS's built-in avconvert.

Same moment, same rectangle, same size

Every encode is compared at second 60, in the middle of a slide that does not move, so a difference comes from the encoder and not from a cursor in a new spot (my reasoning, I never tested a moving frame).

The score is taken on fixed crops: a strip of 15 px English text with some red and blue words, and a 15 px handout region mixing Chinese and English (taken at second 150, after it scrolls). Anything smaller than 1080p is scaled back up to 1920×1080 first, because that is how people watch it. The SSIM comes from ffmpeg's ssim filter.

The original recording only scores 0.971

The reference is not the 360 MB file but the lossless master it was encoded from. Against that master, the 12 Mbps "original" scores 0.971 on the English strip and 0.9998 on the handout. My guess is the colored words drag the strip down, but I did not isolate that here.

So 0.971 is the ceiling for that crop, not 1.0. HandBrake Fast 1080p30 at 6.0 MB scored 0.969, ffmpeg x264 CRF 23 at 5.2 MB scored 0.970, both right on the ceiling while about 98% smaller.

You will not have a lossless master for your own recordings, so the recording itself becomes the reference, and the score shows what the re-encode lost on top of what you had.

Why 0 to 2 OCR errors are noise

I used an open-source OCR engine with its English model on 250 characters of that strip, counting character edit distance against what it read off the master. The untouched file had 1 error. The avconvert HEVC preset and ffmpeg CRF 23 had 0. Dropping to 10 fps kept SSIM at 0.971 and still gave 2 errors.

Then the 720p encodes sat in the same band:

• the avconvert 720p preset at 0.883 with 1
• HandBrake Fast 720p30 at 0.909 with 2

So 0 to 2 is the engine wobbling on this text, and it only tells you a machine can still read the line.

The one encode that broke out came from ImgIng https://imging.ai/, a video compressor that runs locally in a desktop browser and supports batches and H.265: its size-first preset shrank the video to 1080×608 at 24 fps, 10.8 MB, SSIM 0.841 and 7 errors, while its smart-recommend preset in H.264 gave 20.2 MB, 0.969 and 0 errors.

What 0.97, 0.88 and 0.84 look like at 15 px

Zoomed 2× side by side, the 0.969 rows look the same as the original. At 0.883 every word is still easy to read, but strokes turn grayer and fuzzier and the red and blue words lose some punch. At 0.841 letters break up: the dot on a lowercase i fades, so "into" and "it" start to look like they begin with a capital I, and the green Chinese line goes blotchy.

The handout crop of the avconvert 720p preset scored 0.849, but that region isn't in my side-by-side, so I can't say how it looked. For this 15 px strip, high 0.8s meant soft but readable, low 0.8s meant letters starting to break, and 0.97 meant I could not see a change.

The crop also moves the number. On a 20-second clip cut from the master and encoded with x264 medium at CRF 23, the same frame scored 0.924 on a red-only crop, 0.943 on red plus blue, and 0.9993 on black code. So I write the crop and timestamp next to every score, and a strip of colored small text is the first thing I would check.

I still have no threshold, since nobody did a viewer test, and "0.88 is fine" is my opinion about one frame of one clean clip. The routine is short: pick a static frame with your smallest text, crop one gray strip and one colored strip, scale back to source size, compare SSIM against the original, and treat OCR as a floor check that only matters once errors climb clearly past the original's.