I built a compiler that turns computation graphs into the weights of a vanilla transformer - no training anywhere [P]

I've been chasing the question of what algorithms a transformer can actually express - separate from what it can learn. So I built a compiler: define a computation graph in ordinary Python, and it produces the weights of a transformer that executes the graph.

The result is a standard Phi-3-architecture checkpoint that vanilla Hugging Face loads with no custom code and no trust_remote_code. Zero training in the pipeline.

Write-up (origin + how the constructions work):
https://ood.dev/posts/torchwright-intro/

Repo (twelve runnable examples):
https://github.com/physicsrob/torchwright

Background and Motivation

Hand-built transformer weights aren't a new idea. RASP defines a language whose primitives map onto transformer sublayers, and Tracr compiles RASP programs into actual weights. I wanted two things they don't aim for:

• Expressing a computation graph in ordinary Python.
• Targeting a stock architecture, so the output loads in vanilla Hugging Face with no custom code.