ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level [P]

ExTernD: Expanded-Rank Ternary Decomposition

The core idea is that we cannot have ternary PTQ with fixed matrix size - trying to do that is a dead end. So I tried decomposing the matrix to 2 ternary matrices and an inner diagonal scaling matrix.

Now that the inner rank can be arbitrarily large, the accuracy can be arbitrarily small. And it's not that it has to be very large either - I also showed that it takes only slightly more VRAM than current quantization methods. The slight extra VRAM is worth it if we abuse the ternary math.

Key Innovation

• Decomposes weight matrices into two ternary matrices and one diagonal scaling matrix
• Inner rank can be expanded arbitrarily to approach any desired accuracy level
• Memory overhead is minimal compared to standard quantization approaches

Performance Characteristics

• Accuracy approaches that of any quantization level through rank expansion
• VRAM usage is only slightly higher than existing quantization methods
• Leverages ternary math optimizations to justify the marginal memory cost

For full details, see the paper at: https://arxiv.org/pdf/2607.13511