Modders unlock hidden AI power on Nvidia RTX 20-, 30-, and 40-series GPUs, nearly doubling some benchmarks

Sounding off

A used RTX 3090 has become the go-to card for running AI models at home, mostly because 24GB of memory costs far less on a gaming card than on a workstation one. That bargain comes with a catch Nvidia prints in its own spec sheets, though. On GeForce cards, one of the most common kinds of AI math runs at exactly half speed.

A few open-source projects now claim to remove that limitation on RTX 20-, 30-, and 40-series cards.

The limit lives in the tensor cores

When they multiply FP16 or BF16 numbers and add up the results in higher-precision FP32, GeForce cards issue those instructions at half the rate of the FP16-accumulate mode. That FP32 mode is the one most training and plenty of inference actually rely on.

The workstation cards Nvidia builds on the same chips - the RTX A6000, for example - run it at full speed.

The first fix

The first fix came from GitHub user Zagitta, who published gpu-unlock for RTX 30-series cards. It's a Linux kernel module that kicks in before Nvidia's driver loads. That's when it rewrites two speed-select registers on the GPU and resets the card. Nvidia's regular driver then takes over as usual, so nobody has to flash a VBIOS or patch a driver.

On an RTX 3090 Ti capped at 450W, the developer's matrix-multiply tests show BF16 throughput climbing from 87.5 to 157.4 TFLOPS. TF32 nearly doubled, too. FP16 saw a smaller gain of 35%.

Other developers have since adapted the module for the remaining two generations. A version called gpu-unlock-ada doubled an RTX 4090's FP16 instruction rate from 165.6 to 331.6 TFLOPS. That's a raw issue rate, though - the README is upfront about it - so real matrix workloads on a card tested by an independent owner saw smaller gains of 12% to 34%.

On the older side, turing-unlock targets the RTX 2080 Ti and was tested on one of the 22GB memory-modded cards popular with budget AI builders. Its developer measured 36% to 64% faster FP16 matrix math, depending on matrix size. BF16 doesn't apply here, since Turing never had hardware for it.

What benefits most

Unfortunately, gamers won't really notice any difference, since games barely touch this kind of math. Chatbot token speeds hardly move either, mostly because generating text is limited by memory bandwidth rather than raw compute. Prompt processing reportedly runs 14% to 36% faster, though.

Image generation and fine-tuning should benefit the most, since both lean heavily on FP32-accumulate tensor math.

For now, support is largely limited to Linux, apart from a couple of experimental Windows ports validated on specific hardware.

Zagitta considers hardware damage very unlikely, pointing out that the card's usual throttling and power management stay in place. The turing-unlock README is more cautious, warning that tensor cores draw a lot of power and that running them flat out may damage hardware.