"of" after nouns like eve, family, modes
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 0 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the word "of" when it follows a noun that usually takes it, such as "on the eve of", "a family of", "modes of" or "heirs of". Turning it up didn't produce a clear shift in the model's writing.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
named, siblings of the parents were named after, especially the ones who had died already.⏎It is
-
2006⏎Part I: The Umayyad Caliphate (661 – 750 CE)⏎5 Tibet on the Eve of the Arrival of the First Muslim Teacher⏎When al-Salit bin-Ab
-
.⏎The problem is, at least in part, topographical in nature. West Africa’s geography is dangerous, vast, and difficult to subordinate. On the eve of much of West Africa’s independence from France in 1961, R
-
heirs of their cultural⏎heritage. Therefore, present-day Chaldeans and Assy
-
-Baptiste Barrelon (18th October 1818 – 31st September 1885) was born in a family of locksmith
-
of the 328th who felt Sergeant Bernard Early and other members of the patrol deserved credit, too.⏎On the eve of York’s’ return to America in late May of that year, Major General George
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: I have been working on this project, and I have been thinking about it a lot. It's a bit of a problem that I have been trying to fix, but I haven't really fixed it. I have been thinking about it,
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
1900 the population had risen to 4.5 million. Of course all these people needed to get around the ever expanding city and so more modes of transport started to appear.⏎Level 2 covers these transportations issues during 1
-
of dementias on the brain, and the impact on an individual's cognitive function. A dedicated chapter explains how a diagnosis is reached, including the different modes of assessment, from blood tests carried out by a GP, to specialist scans, highlighting
-
of 9, Gabon has a rating of 4, and Malawai has a rating of 1
-
odic retrospective memory, even without any signs of dementia.⏎Given these findings, clinicians can help even healthy e-4 carriers to
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
None: this is the first layer.
Feeds later layers
- L23:1217 weakly excites
- L26:1322 weakly suppresses
- L25:1841 weakly suppresses
- L19:1615 weakly suppresses
- L21:1979 weakly excites the number 5 in a numbered list
- L20:899 weakly suppresses
- L26:1415 weakly suppresses 're' prefix fragment
- L19:1696 weakly excites
- L25:961 weakly suppresses years and numbers before the digits
- L17:1416 weakly excites
- L16:967 weakly excites
- L26:2865 weakly excites
- L21:2406 weakly suppresses
- L25:2555 weakly suppresses
- L18:402 weakly suppresses
- L25:2006 weakly suppresses
- L27:242 weakly suppresses
- L20:2446 weakly excites
- L20:2217 weakly suppresses 'trying' after 'were' or 'are'
- L18:666 weakly suppresses
- L12:1386 weakly suppresses
- L4:1973 weakly excites
- L26:308 weakly suppresses
- L9:1508 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- much
- dx
- thèse
- interchangeable
- ��
- half
- Compare
- 一致
- plement
- ần
Suppresses
- onium
- 到现在
- utenberg
- ования
- ocene
- undert
- 的动作
- omba
- 桷
- ullah