The zkML Bottleneck Isn't That Transformers Can't Be Proved

Why proving modern AI on-chain forced cryptography to get architecture-aware, and what that means for anyone building AI agents on Ethereum. Every deep learning curriculum teaches roughly the same progression: perceptron, MLP, CNN, RNN/LSTM, attention, Transformer, ViT, CLIP. It's usually taught as a story about capability. Each generation gives models a better way to represent information, capture dependencies, or work across modalities. There's another story hidden in that same progression: verifiability. As models got more capable, their computations also got more complex. That distinction only starts to matter once a model's output stops being something a human just reads, and becomes something another system has to act on. Picture an AI agent that can trade an asset, release a payment, authorize a transaction, or choose a robotic action. The question stops being "did the model produce an answer" and becomes: How can I verify that this exact model, with these exact parameters, actually produced this result from this input? That's the problem zkML (zero-knowledge machine learning) exists to solve. And the interesting story in 2026 isn't that Transformers are impossible to prove. They aren't. The interesting story is that proving them efficiently forced cryptographic systems to become aware of how modern AI models are actually built, rather than treating every model as a generic pile of math. The problem is verifiable inference, not inference Running a model is easy. The hard part is proving the result actually came from the computation the protocol expected. A normal AI pipeline looks like this: Input → Model → Output If an application just needs an answer displayed to a person, that's enough. But once the output controls money, permissions, or a physical action, you need another layer: Input → Model → Decision → Verification → On-chain action A smart contract can verify signatures, balances, and deterministic state transitions on its own. It cannot automatically know whether the claimed model actually produced the output, whether the correct weights were used, or whether the operator running the model just fabricated a plausible-looking result. Without an additional verification mechanism, the protocol is trusting the machine that ran the model, full stop. Fine for a lot of ordinary applications. A very different problem the moment an AI agent gets to act on a protocol's behalf. What zkML actually proves At a high level, zkML tries to prove one statement: this model, given this input, produced this output. Instead of asking the verifier to re-run the entire model, the prover runs it once and generates a cryptographic proof of the computation. The verifier checks the compact proof instead of repeating the inference. The hard part is turning a machine learning computation into something a proving system can efficiently handle. Different zkML systems make different trade-offs here. Some compile a model's computation graph into arithmetic circuits. Others use techniques like sumcheck or lookup arguments to handle large matrix operations and non-linear functions more efficiently. Most of this article is really about one question: why does the architecture of the AI model matter so much to the cost of proving it? Why ordinary ML arithmetic doesn't translate cleanly into ZK Machine learning models are built around operations that are convenient for GPUs: floating-point multiplication, matrix operations, normalization, non-linear activation functions. Cryptographic proving systems work differently. They're built around algebraic operations that can be efficiently constrained and checked. So before a model can be proven, its computation has to be re-expressed in a form the proving system actually understands. Some operations translate cleanly. Others don't. That translation gap is a big reason zkML can be dramatically more expensive than ordinary inference. Large matrix multiplications are costly but manageable. Non-linear functions like softmax, GELU, and LayerNorm tend to introduce disproportionate extra proving overhead relative to how cheap they are to run on ordinary hardware. Where quantization helps, and where it doesn't. Quantization swaps high-precision weights and activations for lower-precision representations, which usually makes the computation cheaper to prove. But there's no universal accuracy penalty and no single "best" bit width; the effect depends heavily on the model, the task, and the calibration method used. It's also worth separating two different claims that get conflated a lot: proving that a quantized model produced a given output is not the same as proving that the quantized model behaves identically to the original floating-point model. Those are genuinely different statements, and only the first one is what a typical zk proof actually gives you. Re-reading the architecture progression through proving cost The evolution of deep learning can also be read as an evolution in the structure that a cryptographic prover has to handle. Perceptron and small MLPs Early zkML experiments focused heavily on relatively regular feed-forward networks. Modulus Labs' 2023 The Cost of Intelligence benchmark explored this regime across several proving systems and reported proving an 18-million-parameter MLP in roughly 50 seconds with Plonky2 under its benchmark conditions. That result is important historically, but it is often misquoted. It was an MLP experiment, not a universal limit for CNNs or zkML. CNNs CNNs apply the same convolutional operations repeatedly across an image. That regularity makes their computation relatively structured from a proving perspective. CNNs therefore became an important next target for practical zkML systems. RNNs and LSTMs RNNs introduce something different: sequential dependency. Each state depends on the previous state: hₜ = f(hₜ₋₁, xₜ) LSTMs add several gates and non-linear operations on top of that dependency. This makes the computation less naturally parallel than a CNN and adds more operations that have to be represented inside the proof. Transformers Transformers change the problem again. Self-attention allows every token to interact with other tokens: Q = XWQ K = XWK V = XWV Attention = softmax(QKᵀ / √dₖ)V The challenge is not softmax alone. A Transformer combines large matrix multiplications, global token interactions, normalization, non-linear functions, residual connections, and many repeated layers. That combination creates a much larger and more complex proving workload. ViT and CLIP Vision Transformers apply the same basic Transformer machinery to image patches, while CLIP combines image and text encoders in a shared embedding space. So the proving problem does not disappear when the modality changes. Instead, the computational graph becomes even more interesting: more encoders, more representations, and potentially more components that need to be bound to the proof. The broader lesson is simple: Every architectural improvement in AI changes the shape of the computation that a prover has to understand. It's not just about parameter count A common mistake is treating parameter count as the main predictor of proving cost. It isn't. Two models with identical parameter counts can have wildly different proving costs. A more useful mental model: Proving cost ≈ model size + operation types + non-linearities + precision + memory requirements + architecture + proving system A generic compiler treats a model as a pile of unrelated operations. A proving system designed around a model's actual structure can exploit repetition, sparsity, and natural layer boundaries instead. That difference is exactly what changed the field between 2023 and 2026. The real question isn't "how big is the model," it's "how does the model compute." The 2023-2024 baseline that exposed the bottleneck Before looking at what changed, it's worth seeing what the problem looked like a couple of years earlier. A 2024 survey of decentralized inference methods reported the following latency figures across different verification approaches: | Approach | Reported latency | Model | Guarantee | |---|---|---|---| | Proof of Quality (lightweight assessor) | ~50 ms | MobileNet v2 | Statistical plausibility, under 70% accuracy at that speed | | TEE (Intel SGX) | ~230 ms | VGG-16 | Hardware attestation | | Homomorphic encryption | ~788 s | SqueezeNet | Cryptographic hardness of the encryption scheme | | Halo2 ZK-SNARK | ~2,457 s | MobileNet v2 | Cryptographic soundness | | Optimistic ML (opML) | ~3.6 hours (incl. challenge window) | ResNet-50 | Economic incentives + a challenger | | General ZKML | 24+ hours | ResNet-50 scale | Cryptographic soundness, severe latency cost | These numbers come from different implementations, hardware, and proof statements, so treat them as directional rather than a controlled comparison. What matters is the scale of the problem they show: even proving CNN-level inference could take minutes to hours depending on the approach, and full Transformer-scale proving simply wasn't being reported as a practical, working regime at all. The bottleneck moved For a long time, the main problem in zkML was simply scale. Large neural networks could be represented as circuits in principle, but proving those circuits was often far too expensive to be practical. The important change between 2024 and 2026 was that researchers stopped treating the model as just a generic computation graph. Instead, they began designing proving systems around the structure of the model itself. That shift produced several very different approaches. DeepProve: proving the Transformer directly DeepProve, developed by Lagrange Labs, uses a sumcheck/GKR-based approach rather than relying entirely on traditional arithmetic-circuit representations. The important idea is not just that it uses a different proof system. It is that the prover can take advantage of the regular structure of neural-network computation, including lar