Sampling rate is a correctness property, not a performance knob

In 1997 a Pokémon episode aired in Japan with a four-second red-and-blue strobe. 685 children went to hospital that night. The regulations that followed - Ofcom Broadcasting Code 2.12, ITU-R BT.1702 - are why UK and Japanese broadcast deliveries have to clear a photosensitive-epilepsy check before transmission. I built a screening tool for that check. Live at flashframe-production.up.railway.app, code at github.com/edycutjong/flashframe. Three synthetic test clips ship with it, so you can click one and watch it run without an upload or an API key. This post isn't about the tool. It's about the fact that I shipped two completely different bugs that turned out to be the same bug, and I didn't recognise the second one even after fixing the first. The rule is a window function The regulation says: no more than three flashes in any one second. That's not a video-processing problem, it's a time-series query. Per-frame luminance goes into ClickHouse, and the detection is windowed SQL - pair opposing luminance transitions into flashes, count them over a sliding one-second window: sum(is_flash) OVER ( PARTITION BY tile ORDER BY frame_idx ROWS BETWEEN {int(fps)-1} PRECEDING AND CURRENT ROW ) AS window_flashes Look at that window size. It's fps - 1 rows, because "one second" of frames depends entirely on how many frames per second you sampled. That parameter is about to be the whole story. Over a 92-minute feature - 138,240 frames - this runs at p50 106 ms, p95 163 ms. Query time only; ffmpeg extraction and model adjudication are outside the timed region, and I report ingest separately because it's dominated by transport, not the database. Bug one: a safe clip that failed Extraction ran at 10 fps. Cheap, and plenty for a rule about events at 3 Hz. One of my test clips alternates 5 times a second - under the limit, should pass. It came back at 2.08 flashes/sec and got flagged as a violation. At 10 fps sampling, the Nyquist limit is 5 Hz. A 5-alternation-per-second flash sits exactly at that boundary. The samples land at a beat frequency against the actual flashing and you measure something that was never there. Classic aliasing - the wagon-wheel effect, except the wheel is a seizure hazard and the wrong answer ships in a compliance report. The fix wasn't to sample everything at 60 fps. On a 90-minute feature that's 6× the extraction and 6× the rows for a property that matters in maybe 30 frames out of 138,240. Instead the agent re-samples only the span whose verdict is uncertain: async def resample_frames(frame_start: int, frame_end: int, target_fps: int) -> dict: nonlocal resample_count, current_measured_rate if resample_count >= 2: return {"status": "error", "message": "Max resample iterations reached."} resample_count += 1 scan_id_new = run_extraction(video_path, fps_override=target_fps, frame_start=frame_start, frame_end=frame_end) await setup_db_and_ingest(run_query_tool, scan_id_new, video_path, 25.0, target_fps) new_res = await detect_violations(run_query_tool, scan_id_new, fps=target_fps) It's a tool the model calls on its own when a result lands near the threshold. In the actual run: Flagged span 1025-1055 (2.0833333333333335 flashes/sec)... >>> resample_frames(span, 30) >> resample_frames(span, 60) >> adjudicate(1025, 1055) <<< At full rate the clip resolves to 2.82 flashes/sec - under the 3.00 limit. It passes. A single-pass tool reports a false failure on it. The same mechanism caught the opposite error on a different clip: a genuine strobe read 5.0 at 10 fps and resolved to 6.25 after escalation. Undersampling had understated a real hazard. That direction is the one that actually hurts someone. Bug two: the model couldn't see the strobe The SQL finds candidates; a multimodal model looks at the frames and says what the flashing thing actually is. I clip the exact span with ffmpeg and hand it over. It kept returning "no significant flashing." On a clip that is nothing but full-screen black-white alternation at 6.25 Hz. I assumed a prompt problem and spent real time rewriting the prompt. It was not a prompt problem. Video sent to the API is sampled at 1 fps by default. A 6.25 Hz strobe sampled once per second isn't faint in the frames the model receives - it is absent. Every frame it saw was a still. types.Part( inline_data=types.Blob(data=clip, mime_type="video/mp4"), video_metadata=types.VideoMetadata(fps=24) ) One parameter. Same test, failed at the default and passed at 24 fps. Two subsystems, two days apart, one root cause - and I still spent hours on the second one hunting a prompt bug, because the first had presented as a SQL bug. The category I'd filed the first fix under was "SQL window sizing," which is exactly the wrong abstraction level to have learned it at. The thing worth stealing Sampling rate is a correctness property, not a performance knob. When you downsample, you're not accepting a slightly blurrier version of the truth. Above the Nyquist limit you get a confidently wrong answer with no signal that anything is off. Neither bug threw an error. Both produced clean, plausible numbers. If any part of your pipeline samples a signal - video frames, metrics, sensor reads, log aggregation windows, an LLM's view of a time series - write down the highest frequency that matters and check you're sampling above twice it. That question took me two bugs to learn to ask. What this doesn't do It's a screening-grade pre-check, not a certified lab test, and it says so on the report and in the certificate. It measures luma code value from the decoded signal under an assumed reference display; a certified test measures photometric luminance at a calibrated one. Screen area is a 3×3 tiled proxy, not per-pixel. Measured accuracy against constructed ground truth is exact on the full-field case and +12.9% on a small-area case, biased toward over-reporting. I disclosed that rather than adding a correction constant - the entire claim is that thresholds come from published criteria rather than from whatever made the demo look right, and a fitted constant would destroy it. One more thing I got wrong and had to walk back: the certificate was writing the model's estimate into the measured field. Two runs of the identical clip produced certificates reading 6.25 and 5.0. A certificate whose headline number moves between runs is not a certificate. The measured value now always comes from the SQL, and the model's estimate is stored separately - the database measures, the model judges, and the report labels which is which. All the test footage is synthetic, generated by a script in the repo. Clone it and re-derive every number above: git clone https://github.com/edycutjong/flashframe.git cd flashframe uv pip install -e . -r requirements.txt python generator.py # regenerate the seed clips python bench.py # re-run the 138,240-frame benchmark Live demo: https://flashframe-production.up.railway.app Top comments (0)