How a language model works, one neuron at a time

Step 1 of 7

A language model reads text one piece at a time.

Before the model can do anything with a sentence, it cuts it into pieces called tokens: whole words, parts of words, spaces and punctuation. Here is a real sentence from the web text the model read for this site, cut the way the model cuts it.

Step 2 of 7

Inside, the text passes through a stack of layers.

Each token travels up through the layers in turn. Every layer reads what the layers below wrote and adds its own notes, so by the top the model has built up a rich picture of the text and can guess what comes next.

Each layer has thousands of tiny dials called neurons.

This is one layer seen face-on. Every dot is a neuron. Neurons that write similar things sit close together. The brighter dots are the ones whose behaviour looks readable; the ringed one is the neuron we follow next.

Here's one neuron reading a sentence.

The same sentence, now seen through that one neuron. Amber shows where it fires, its activation: hardly at all on most tokens, hard on one. Claude read this neuron's strongest moments and called it gemstones and jewelry words.

Step 5 of 7

What happens if we turn it up?

We can add this neuron's signature to the model's thinking while it writes, and make it louder or quieter. That is steering. Drag the slider: a small push and the model starts talking about rings and carats; a big push and it falls apart. Nothing runs in your browser, every text was written ahead of time.

Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.

Steering strength turned down a lot

Prompt

Here is something I have been thinking about lately: The weather this morning was My grandmother always used to say that In the news today,

Here is something I have been thinking about lately: I have a new house in the state of the US, and I need to understand the pressure of the US government for the immediate (preference) and a certain (long-term) in the short and the in the long terms. The

Step 6 of 7

Most neurons aren't this clear.

The gem neuron is a lucky find. Most neurons respond to several unrelated things at once, or to something so small it is hard to name. Here is a typical one from the same model, with the label Claude gave it. Every label on this site is a guess from limited evidence, which is why the evidence is always shown next to it.

Why this is hard, in a little more depth.

unclear: weak, articles before nouns

This neuron gives only weak responses, mostly on small words like 'the', 'a' or 'some' that come just before a noun in the middle of a sentence. It does not clearly change the writing when turned up or down.

#1 peak

organize and prioritize the goals to be accomplished.⏎Popular methods of time management practiced in the 21
#2 peak

some extra time to make a decision. It now hopes to make a launch decision on Monday morning.⏎“We have
#3 peak

One of the most important requirements is to sample where irradiation and oxidation are minimized, either by

Step 7 of 7

Now go explore.

You now know what a token, a layer, a neuron and steering are, which is everything the rest of the site assumes. The map shows all eighty-six thousand neurons at once; every dot opens a page like the ones you just saw.

Open the mapall layers, all neurons
The gem neuron's pageeverything we measured about it
Guess the neuronread steered text, pick the label
Learnglossary and short explainers
The gem neuron on the mapwith its connections drawn