DEV Community

Every despill formula bets on colours your subject isn't wearing

A despill step is usually one line of maths, picked from whichever tutorial came first. To see what the three common lines cost, the comparison here uses a self-made 1280×720 test card with a green screen at RGB(20,175,65) and ten patches: hair strands, a white shirt, a light-blue shirt, light and dark skin, a yellow object and a grey veil at α=0.45.

Clean foreground and alpha are known. The spill is not real light; it was added with a model that peaks at the outline, so the numbers describe this card only. Results are composited on grey (110,110,110) with the true alpha and compared to the clean version as ΔE76, hue as the HSV angle. One Apple-silicon Mac, Python and NumPy, FFmpeg for one cross-check, single run.

Spill sits on the outline more than inside

Uncorrected, the hair strands carried 30.7 of extra green, ΔE 24.0. The veil at α=0.45 came out with G about 41 above R and B, ΔE 28.8.

The white shirt read ΔE 16.6 on its 6-pixel edge and 7.8 inside. Light skin was the worst case: its edge moved from hue 23° to 70°, ΔE 34.5, against 11.7 in the interior.

Per pixel, a despill formula has to decide whether this green came from the screen or the object. It can't know, so it assumes.

Average-limit assumes nothing is greener than its neighbours

G = min(G, (R+B)/2) was the cleanest on almost everything: hair ΔE 6.2, veil 6.5, light-blue shirt 2.7. Its bet is that no real colour in the shot has green above the mean of red and blue, which yellow breaks by definition.

The yellow patch, clean at (245,205,30), went from (248,228,39) with spill to (248,143,39) after the formula, hue 49° to 30°, ΔE 39.4. Untouched it was 12.1. Yellow-greens and pale cyans sit on the same side of the line.

FFmpeg's despill filter at its defaults (mix 0.5, expand 0) matched this formula within 1 per pixel on this card. Its source wasn't read for this, so the claim stops at "almost the same on this card".

Max-limit assumes spill always pushes green to the top

G = min(G, max(R,B)) is the cautious one: yellow and skin interiors were not touched at all. That's its failure too.

The light-blue shirt, clean at (196,214,236), picked up spill to (198,237,244). Green rose by 23, but it stayed under blue at 244, so the formula saw nothing to remove and left ΔE 12.3 exactly as it was. The light-skin edge stayed at (227,227,168), hue 60°, ΔE 27.6.

The hidden bet is that spill only exists once green becomes the largest channel, and on pale blue or warm skin it can add a lot without winning.

Complement-fill assumes the lost energy belongs to magenta

The third family gives some of the removed green back to red and blue so the pixel doesn't darken. The variant used here takes s = max(0, G − (R+B)/2), subtracts s from green and adds s/2 to each of red and blue.

Brightness held up, but the refill overshot. The light-skin edge came out (246,197,188), hue 23° to 9°, pink. The veil and the white shirt edge drifted toward purple, hues between 256° and 286°. Yellow was the worst result in the whole table, ΔE 50.8 inside and 54.3 on its edge.

The refill ratio wasn't swept, so whether some split keeps skin warm without hurting yellow is still open.

One drift sits outside the formulas: after average-limit, half-transparent areas such as the hair strands landed on a blue-grey near hue 210°, because the screen has B=65 and R=20 and removing its green leaves the blue behind. That's a missing step, subtracting the screen colour first, not an assumption of any formula.

Choose the formula per shot as a policy

Formula What it assumes What broke on this card
Average-limit Nothing real is greener than (R+B)/2 Yellow 49° to 30°, ΔE 39.4
Max-limit Spill always makes green the top channel Light-blue shirt untouched, ΔE 12.3; skin edge ΔE 27.6
Complement-fill (s/2 to R and B) Removed green should return as magenta Skin edge 23° to 9°; yellow ΔE 50.8

Three years on codecs taught me to read every rounding rule as a choice of which error to ship, and this table reads the same way. Yellow props mean average-limit, and FFmpeg's default with it, will damage them, so max-limit or a mask is safer. Pale blue clothes or light skin under heavy spill mean max-limit leaves the green alone. Complement-fill trades skin tone for brightness.

I'd record the choice per shot next to the key settings rather than bake one line into the pipeline. The step belongs after alpha and before compositing; run before the key, it turned the screen dark blue-grey and the keyer stopped seeing it.

AI matting doesn't remove the decision either. A real green-screen clip went through the video matting Beta of ImgIng https://imging.ai/ in a desktop Chromium-based browser, with the model running locally. Composited over grey at frame 45, its transparent WebM still had green-leaning pixels at the hair tips and on the hand, from 2.7% to 14.8% depending on region and on which of its two models produced the matte; average-limit took every one of those counts to zero.

That module isn't mine. My reading is that the model returns transparency and the colour usually comes from the source frame, so the assumption is still yours to pick.

Credit for the real clip goes to VOA Learning English via Wikimedia Commons (public domain; trimmed and re-encoded here). The test card is self-made, with spill added by a model.

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.