"of" in "most X of all" superlatives
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 20 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron responds to the word "of" in phrases like "worst of all", "strangest of all" or "the most ... of any state", where a superlative is about to be completed. It turns slightly negative on "very" after "the". Turning it up made the model stutter "of all of all of all", which is the same phrase it fires on.
Written by Claude from this neuron's activations and steering outputs. Confidence: high. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
alligator wasn’t feeling⏎His feet felt draggy.⏎His skin felt baggy.⏎His tail wouldn’t swish this way and that.⏎And, worst of all, his big jaw wouldn’t SNAP.⏎Only two lines actually rhymed
-
and the telescope) can see. Those objects are getting farther away all the time because the universe is expanding, and moreover the expansion is accelerating. Strangest of all, the luminous matter of the galaxies—everything we see shining in the
-
to find brief and simple answers that are appropriate to their questions; answers that they can understand and that do not overwhelm them with too many words.⏎Perhaps most difficult of all, it involves examining our own feelings and beliefs so that we can talk to
-
severe cases<|endoftext|>Considering the fact that half of you reading this right now are probably wearing one, It’s only right to start this week off with the most commonplace of all acronymic items– the OCBD (oxford cloth button down.) While
-
invented in the 19th century. While it is possible to get food-borne illnesses from many different foods, raw milk is one of the riskiest of all, according to the Centers for Disease Control and Prevention.⏎
-
iamond⏎DIAMOND, a mineral universally recognized as chief among precious stones; it is the hardest, the most imperishable, and also the most brilliant of minerals. These qualities alone have made it supreme as a
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: what is the most efficient way to model a system where a person could say a message in a room, say, A most efficient model would be a language, which is a representation, which can be used to model both the message and the
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
and philosophical principles, this is a thorough exploration of the concept and practice of⏎liberalism.ABOUT THE SERIES: The Very Short Introductions series from Oxford University Press contains hundreds of titles in almost every
-
12 percent of the nation’s women and girls. That steady hum of progress in Dhaka--despite its unscrupulousness--has helped the very poorest break free of poverty.⏎Aside from poverty alleviation, many also claim that
-
system, and multithreading.⏎Also included is coverage of some of Java's most powerful features,⏎such as generics, lambda expressions, and the very important⏎modules feature. A number of recent additions to the Java language, such
-
two complement each other. In simple terms, these two creatures do not have the same nutritional requirements which means, they cannot be fed with the same food. The very small breed rabbits may be okay to
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L17:1771 excites
- L19:2622 excites
- L13:1659 weakly suppresses
- L16:961 weakly excites
- L18:2347 weakly suppresses
- L18:3048 weakly suppresses
- L19:2036 weakly suppresses
- L17:2241 weakly excites the word 'very'
- L17:152 weakly suppresses mid-word fragments, often 'intric'
- L19:3054 weakly excites
- L18:2241 weakly excites
- L18:1631 weakly excites
- L15:1196 weakly excites
- L5:729 weakly suppresses
- L8:1074 weakly excites
- L13:1249 weakly excites
- L17:459 weakly suppresses
- L8:506 weakly suppresses
- L18:1619 weakly suppresses
- L11:223 weakly suppresses
- L19:715 weakly excites
- L6:2430 weakly suppresses
- L7:2266 weakly excites
- L18:1174 weakly suppresses
Feeds later layers
- L23:567 strongly excites
- L26:711 strongly excites 'years' after an age number
- L27:447 strongly excites
- L25:2020 strongly excites
- L21:1718 excites
- L22:142 suppresses 'of' after a quantity or noun phrase
- L26:279 suppresses
- L23:258 excites
- L26:920 excites
- L26:115 suppresses
- L22:1993 excites
- L22:288 excites
- L27:769 excites
- L23:2665 excites
- L22:477 excites
- L21:2802 suppresses 'of' inside set phrases like 'course of'
- L23:1712 excites
- L24:2753 suppresses
- L21:2646 excites
- L24:860 excites
- L26:1052 weakly excites
- L25:1241 weakly excites
- L27:658 weakly excites
- L21:1306 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- of
- _of
- Of
- Of
- of
- OfFile
- →of
- _Of
- OfString
- _OF
Suppresses
- ever
- umont
- greatest
- 一个是
- iclass
- 首家
- -Saharan
- EVER
- highest
- 쉽