TokenPrint: An Open-Source 3D Visual Debugger for LLMs and Transformers

TokenPrint: An Open-Source 3D Visual Debugger for LLMs and Transformers

Large language models are usually presented as a stack of equations, architecture diagrams, and source code. That explains what the model is. But it doesn't always explain what is actually happening to a token as it moves through the model.

What is TokenPrint?

TokenPrint is an open-source interactive visualization and debugging environment for transformer and LLM inference. The goal is simple: Make the internal computation of a transformer something you can actually see and inspect.

Exploring the Transformer Computation

Instead of looking at a linear sequence of operations like x → attention → MLP → x, you can explore the individual operations that produce that result. You can move through the model layer by layer and inspect things such as:

• Token embeddings
• Positional information
• RMSNorm
• Q, K, and V projections
• Grouped-Query Attention
• RoPE
• Attention scores
• Softmax
• Weighted value aggregation
• Output projection
• SwiGLU MLP projections
• Residual streams
• Logits and predictions

Following a Token through the Model

One of the ideas behind TokenPrint is to treat inference as a journey. A token starts as a discrete token ID. It becomes an embedding. That representation enters the first transformer layer. Then it passes through normalization, attention, projections, nonlinear transformations, and residual connections. The representation continues evolving through the remaining layers until the model produces its final logits.

Why 3D?

A transformer is already a computational graph. The problem is that a traditional diagram becomes difficult to read once you start showing many layers, branches, and intermediate tensors. TokenPrint uses 3D space to make those relationships spatial.

Inspecting Individual Components

Selecting a component opens an inspection view containing information such as:

• What it does: A plain-language explanation of the operation.
• Equation: The mathematical operation represented by the component.
• Dimensions: Input and output dimensions.
• Parameters: The number of learned parameters involved.
• Model data: The actual tensor path associated with the component.
• Provenance: Whether the displayed information is real model data, derived information, conceptual structure, or simulation.

Real Model Data

A major design goal of TokenPrint is to distinguish between what the model actually provides and what is only being illustrated. The visualizer uses explicit provenance categories:

• REAL: Directly obtained from the model/runtime.
• DERIVED: Calculated from real model information.
• CONCEPTUAL: An educational representation of a model concept.
• SIMULATION: Intentionally simulated behavior.

Exploring Tensors

TokenPrint also provides a tensor inspection workflow. Instead of treating tensors as invisible implementation details, you can inspect their metadata directly.

Learn while Exploring

TokenPrint is also intended to be an educational tool. A component should answer more than: "What is this box?" It should explain:

• What does this operation compute?
• Why does the transformer need it?
• What is its input?
• What is its output?
• How does it connect to the next operation?
• Which parameters and tensors are involved?

Built for Experimentation

The long-term goal is not only visualization. TokenPrint is being designed as an environment for inspecting and experimenting with transformer computation. That includes areas such as:

• Attention inspection
• Activation analysis
• Residual stream analysis
• Tensor inspection
• Layer and head experiments
• Ablations
• Activation patching
• Trace replay
• Model comparisons

Open Source

TokenPrint is open source and built for people who want to learn, experiment, research, and contribute. There are opportunities across:

• Frontend and 3D visualization
• Transformer architecture support
• Inference backends
• Tensor instrumentation
• Debugging tools
• Visualization UX
• Educational content
• Research experiments

What's Next?

I'm working toward making TokenPrint much more than a visualization of one local model. Some of the directions include broader Hugging Face model support, better model capability detection, richer execution traces, remote inference, experiment workflows, and shareable transformer traces that can be explored without reproducing the original runtime.