Hacker News

LZ-RDO lands in ARM's astc-encoder

My rate-distortion optimization work for ASTC is now in ARM's astc-encoder (PR #781). Fabian "ryg" Giesen took my original implementation, added a round of optimizations and improvements, and landed it.

If you ship ASTC textures inside Oodle, zlib, zstd, LZ4 or LZMA-compressed packages, your downloads can now get smaller with one command-line flag.

The problem: ASTC looks like noise to LZ

ASTC is a fixed-rate format. Every block is exactly 128 bits, whether it's 4x4 or 12x12. On the GPU that's what you want, but nobody ships raw textures. They go into packages that get compressed with a general-purpose LZ codec.

To an LZ compressor, a normal ASTC bitstream looks almost random: each block's endpoints and weights are fitted to that block's pixels, so neighboring blocks rarely share bytes. That leaves the compressor nothing to match.

RDO fixes this by choosing, among encodings that look nearly as good, the ones that repeat bits the encoder has already emitted. The GPU sees a valid ASTC texture of exactly the same size, and LZ sees matches. It's a quality-for-package-size trade, and a single knob controls it.

How it works

  1. Encode normally, plus a "restricted" encoding

    First the image goes through the regular astcenc encode. With LZ-RDO enabled, each block also gets one extra trial encode using the most common configuration: 1 partition, 1 weight plane, endpoints quantized to at most 32 levels, and only the always-available block modes for 2D. Blocks that share a configuration are much more likely to share bits, so this gives the optimizer more chances to find repeats.

  2. Measure where errors will be visible

    Raw squared error (PSNR) is the wrong measure when you're adding error on purpose. Errors in smooth areas and slow gradients are easy to spot. Errors in noisy texture or near edges are mostly hidden. So before encoding, the image is split into tiles (about 256x256, or 32ยณ for 3D) and each texel gets an activity weight:

  3. Move-to-front histories for endpoints and weights

    ASTC packs weights from the top of the block downward and the mode and endpoints from the bottom up. That makes splitting a block into "endpoint bits" and "weight bits" a simple bitmask. The optimizer keeps two move-to-front lists of recently used values:

  4. Build candidates for each block

    For every block, the encoder scores:

  5. Score with a simple LZ cost model

    Each candidate's cost is cost = weighted_SSD + ฮป ยท estimated_bits

    The bit estimate is deliberately simple: Unmatched bits are billed as literals. ฮป is scaled by texel count, so the same value behaves about the same at 4x4 and at 12x12.

  6. Multiple passes, in parallel

    The image is processed in independent chunks of blocks, so it spreads across threads. Each pass seeds the histories with random blocks from the chunk, and passes alternate between forward and reverse order. Because splicing produces the same bit pattern many different ways, a small hash-table cache of decoded blocks avoids decoding them again during error checks.

Using It

Command line:

astcenc -cl albedo.png albedo.astc 6x6 -thorough -rdo-lambda 20

API:

astcenc_config_init(ASTCENC_PRF_LDR, 6, 6, 1, ASTCENC_PRE_THOROUGH, ASTCENC_FLG_USE_LZ_RDO, &config);
config.lz_rdo_lambda = 20.0f;

Tuning guidance:

Limitations

Results

Thanks

Huge thanks to Fabian for the optimizations, fixes and for getting this merged, to ARM for taking it upstream, and to Charles, Fabian and Sean for the Oodle Texture work this comes from.

2 Comments

Pete Harris 11/17/2026 11:50:19 am

As the Arm end of the landing team on this I just wanted to say thank you, it's great to have this in the public compressor implementation =)

Reply Jon Olick 11/17/2026 12:24:30 pm

You are welcome! :)

Reply Leave a Reply.

Read on Hacker News ↗ ← Back to News

Comments

No comments yet. Start the discussion.