unclear: word before 'first', 'last' or 'further'
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 17 of Qwen3-0.6B. Its strongest snippets are mixed: they peak on different tokens. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on words that often lead into a sequence or key point, like 'the first', 'the last' or 'further'. The pattern is loose and hard to name. Turning it up made the model produce repeated words and half-finished phrases.
Written by Claude from this neuron's activations and steering outputs. Confidence: low. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
animal protein causes a lining of fat to form in their stomach walls. This lining of fat affects their absorption rate which can end up becoming dangerous to them. Aside from containing animal protein, dog food also contains meat.
-
in your professional development<|endoftext|>Mackinder’s Heartland Theory made SIMPLE⏎Do you want to learn about Mackinder’s Heartland Theory? Then you have come to the right
-
PRAYER AND PRAISE (5 minutes)⏎Further info? Click here⏎MEMORY WORK (12 minutes)⏎"You will seek me and find me when you seek me with all your heart" Jeremiah 29:13⏎Write the
-
the “unsinkable” Titanic set sail from Southhampton, England on April 10, 1912.⏎This is one of the last photographs of Titanic taken from the shore as the ship departed towards New York City.⏎
-
the body. The solution contains chemicals such as⏎The mixture of these solutions allows the body to last longer than it could have stayed in normal conditions. They first start by drawing blood from the body using the veins. After this, they inject the
-
because the typical symptoms of an acoustic neuroma are non-specific and can often be put down to other causes. Symptoms include:⏎- Hearing loss (this is the first symptom of an acoustic neuroma in around 90% of cases)⏎-
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: I have a 30000000000000000000000000000000000000000000
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
Zeus, lord of the gods, who rapes his sister Demeter, and the product of that rape is Persephone. They never married. Indeed, Zeus would
-
top scholars discuss the historical Israelites in Egypt and archaeological evidence for and against the historicity of the Exodus.⏎BAR coauthor Lily Singer-Avitz suggests that several finds discovered in the later strata,
-
. At least, that’s what we thought we would be doing. Continue reading →⏎Black pepper and its flavoring companion, salt, can be found in kitchens all around the world. But did you
-
seems appropriate to have some humility around this energy transition,” said co-author David Rapson, an associate professor at the UC Davis Economics Department. “Appro
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L13:1971 suppresses
- L11:1383 excites
- L12:2475 suppresses
- L8:968 excites
- L7:2458 excites
- L16:961 suppresses
- L11:2850 suppresses
- L7:2553 suppresses
- L12:1705 suppresses
- L4:1724 excites
- L14:1373 excites
- L13:2562 excites
- L16:1471 excites
- L10:2453 suppresses 'of' after publisher or division nouns
- L14:623 excites
- L5:2500 excites
- L14:2671 suppresses
- L9:755 weakly suppresses
- L9:906 weakly excites
- L14:526 weakly suppresses
- L4:929 weakly suppresses
- L9:731 weakly excites
- L4:869 weakly excites
- L9:1705 weakly suppresses
Feeds later layers
- L18:1812 suppresses
- L18:976 suppresses
- L18:1672 weakly excites
- L18:919 weakly suppresses
- L19:1235 weakly excites
- L18:2621 weakly suppresses
- L18:759 weakly excites
- L18:1397 weakly excites
- L20:1592 weakly suppresses
- L22:781 weakly suppresses
- L23:553 weakly suppresses
- L21:1974 weakly excites
- L23:2021 weakly excites
- L18:1624 weakly suppresses
- L24:594 weakly suppresses
- L21:1420 weakly excites
- L23:1785 weakly suppresses
- L19:1786 weakly excites
- L20:2830 weakly excites
- L18:456 weakly excites
- L24:1231 weakly excites
- L24:2098 weakly excites
- L19:462 weakly suppresses
- L24:1411 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- au
- adget
- anson
- фор
- rior
- arbon
- 俦
- ốt
- ybrid
- esterday
Suppresses
- ije
- 虱
- elen
- ships
- S
- anj
- primaries
- 舍
- anine
- multipart