← All writing

One transformer block, neuron by neuron

A tiny real model (d = 8, 2 heads, MLP 8 → 32 → 8, 4 blocks) running on your sentence. Every circle holds the number it actually computed. Hover a circle to see the lines feeding it.

Head

negativepositiveLines: weights, orange positive, violet negative, darker means larger.Arrow keys move between steps