Measuring diffusion video performance on a MacBook: one speedup and a large gap

Last month, I published a benchmark showing a 1.125× speedup from block-residual caching on 4-bit FLUX. The main lesson was not the multiplier. It was that my original quality metrics had been measuring the wrong thing, and that acceleration claims often combine speed, trajectory preservation, and perceptual quality into one number.

For the follow-up, I chose a stricter target: real-time autoregressive diffusion video on an Apple M5 Max, with the definition of "real time" frozen before results were visible. The tested configuration did not meet that target. The fastest claim-eligible result was 1.418 native generated frames per second, compared with a 16 FPS target. That is an 11.28× gap. I am publishing the result because the measured bottleneck, one systems improvement, and two rejected hypotheses are useful even without a real-time result.

The evidence can be checked from a repository checkout:

git clone https://github.com/kkjcodes/liveframe
cd liveframe
python -m pip install liveframe
liveframe verify \
  artifacts/liveframe-publication-claims.v1.json \
  --artifacts-root .
liveframe recompute \
  artifacts/liveframe-publication-claims.v1.json

The setup

LiveFrame evaluates Wan2.1-T2V-1.3B-based causal video models across NVIDIA H100 CUDA and Apple M5 Max MLX/Metal. The experiments include:

• Causal Forcing++ for the clean M5 performance fixture
• Rolling Forcing for the CUDA-to-MLX portability study
• Frame-wise Causal Forcing++ for the H100 cache-reuse experiment

The clean M5 fixture produces 81 pixel frames at 480×832, corresponding to