'of' after a title or position noun
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 16 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron responds to the word 'of' when it comes right after a job title, rank or position, as in 'the 38th president of' or 'the 27th state of'. Turning it up did not produce text that clearly showed this, so the effect is unclear.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
January, 2009), a German canton of Switzerland (March, 2009), and the Piemonte-Toscana region of northern/central Italy (images only, February-March, 200
-
Secretary of State Hillary Clinton, the world would have laughed or stood in horror as it listened to a metaphorical dragon.⏎Just how can a candidate to the leadership of the political party, express such<|endoftext|>Presentation on theme: "Case Review What does
-
in his job, from being the Commander-In-Chief to the Guardian of the Economy.⏎Brief biography of Gerald R. Ford, the 38th president of the United States of America.⏎Timeline of the life of former U.S. President
-
superparamagnetic iron oxide nanoparticles (SPIONs), attached to therapeutic cells, show promise. Magnetic devices could be used to move such cells to diseased areas of the body. But current ways of attaching SPIONs to therapeutic cells are difficult
-
in exchange for the American renunciation of any claims on Texas. On March 3, 1845, Florida became the 27th state of the United States of America.⏎Until the mid-twentieth century Florida was the least
-
descent today."⏎On 25 March 2007, bicentenary of the Abolition of the Slave Trade, then Prime Minister Tony Blair of Britain publicly stated that slavery was "a crime against humanity" and offered "deep
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: I have been using a technique to create a model of a specific segment of the segment of the segment, where is that technique based on the segment of the segment of the segment is, I have used the segment of the segment of the segment
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
its children. Visiting a node usually involves transforming it in some way or collecting data from it.⏎In "pre-order traversal", a node is visited __before__ its children. In "post-order" traversal, a node is visited __after
-
This tiny difference of starting values caused the equations to spit out a completely different set of answers. To mathematicians this effect is known as sensitive dependence on initial conditions and it is one of the hallmarks of mathematical chaos. It means
-
and a 38.7% improve in depression throughout the 90 days following lively shooter drills, in contrast with the 90 days earlier than the drills.⏎The authors of this research, together with Mai ElSherief, PhD
-
For each value in the dataset, we pair its right-most digit to the corresponding stem that matches it's left digit. For example, if we had a⏎76 in our dataset, we would pair it with the⏎7 leaf in the
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L14:1785 excites
- L15:1476 suppresses
- L15:447 excites
- L15:1594 suppresses
- L15:408 excites
- L14:739 suppresses 'the' or 'your' before a body part or place
- L15:1627 weakly excites
- L14:543 weakly suppresses
- L15:673 weakly excites
- L13:2925 weakly excites
- L12:162 weakly excites
- L14:1865 weakly suppresses
- L13:289 weakly excites
- L15:1375 weakly excites unclear: mid-sentence function words
- L13:902 weakly excites
- L15:2014 weakly suppresses
- L11:629 weakly excites
- L15:558 weakly suppresses
- L15:1458 weakly suppresses
- L4:669 weakly suppresses
- L11:21 weakly suppresses
- L15:2295 weakly suppresses
- L15:2404 weakly excites
- L7:2232 weakly excites
Feeds later layers
- L17:1888 excites
- L17:2126 excites
- L17:2749 excites
- L20:1038 excites
- L19:2602 suppresses
- L17:2022 excites
- L17:1193 excites
- L17:655 weakly excites
- L17:1460 weakly suppresses
- L20:584 weakly suppresses
- L21:965 weakly excites
- L19:2279 weakly suppresses
- L17:138 weakly excites
- L20:731 weakly suppresses
- L21:666 weakly excites
- L19:1344 weakly excites
- L21:971 weakly suppresses
- L17:1796 weakly suppresses
- L21:920 weakly excites
- L17:489 weakly excites
- L21:2706 weakly excites
- L17:2014 weakly suppresses
- L17:1060 weakly excites
- L18:2108 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- world
- world
- -world
- ORLD
- life
- orld
- Big
- .invokeLater
- _world
- big
Suppresses
- 贝壳
- =form
- ?」
- =out
- :class
- oref
- ---</
- orna
- essian
- `t