'of' after a noun phrase
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 17 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the word "of" when it follows a noun that needs a following phrase, such as "one of the most unique caves of" or "a trading post of". Turning it down made the model repeat "of" over and over, so this neuron seems tied to the word "of".
Written by Claude from this neuron's activations and steering outputs. Confidence: high. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
Here's a link to place your order.⏎The passage above needs a great deal of care and attention because this is one of the most difficult 'shifts' of 2nd finger to get in tune. When you are playing the tune you
-
AD and subsequently belonged to a series of local empires. Modern Singapore was founded in 1819 by Sir Thomas Stamford Raffles as a trading post of the East India Company with permission from the Johor Sultanate. The British obtained
-
. They have been reintroduced to National Reserves of Mongolia and China⏎CONSERVATION STATUS: Classified as Extinct in the
-
different strategies developed by elite men of color after the Haitian revolution to secure their status in the Atlantic World. Considered as “the first truly modern international crisis of exiles,” the
-
round Carolingian shields was the use of a sugarloaf boss in the centre which served to deflect the blows of an opponent and also as a pointed weapon of
-
the trauma of exile and one of the most devastating genocides of modern times.⏎Some 1.5 million Circassians were killed in the Caucasian War of the mid-to-late 19
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: of of of, or else? of or else? of or else? 你也可以吗? 里面我看看看看。 里面好像上你看看看看。 之间还是上一下。
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
449F for pure metal and about 512F for tin-zinc alloys. Tests show that screws plated with the material do not stand much of a chance in hot areas. One test run several years ago by J. B
-
the language?⏎No, I’d say it started out as a Standard English euphemism. It wasn’t much of a semantic leap to take a word which literally means something that roosts and
-
sure that no Earthly gunk makes it off-world—primarily so that it doesn’t contaminate whatever science an Earth-born probe is doing.⏎None of that has ever really been a problem.
-
olysis of human red blood cells would occur if the cell were in .. solution A) isotonic B) hypertonic C) hypotonic D) none of these I think it's B. I'm not too sure though.⏎November
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L16:1088 strongly excites 'of' after a noun phrase, as in 'of course'
- L3:315 excites 'of' after a noun phrase
- L15:785 suppresses 'of' in set phrases like 'of course'
- L4:4 suppresses
- L6:357 suppresses
- L5:363 excites
- L0:2726 excites
- L2:895 suppresses
- L0:1220 suppresses
- L0:40 suppresses 'of' in 'sort of' hedges
- L6:13 excites
- L9:2264 excites 'of' after superlatives and 'the X of'
- L10:180 excites
- L1:2567 excites
- L15:80 suppresses
- L10:469 suppresses 'of' after a noun phrase
- L16:428 weakly excites unclear: mid-sentence commas, 'most' before a verb
- L8:14 weakly excites
- L1:725 weakly suppresses "to" in "referred to"
- L16:41 weakly suppresses
- L5:1896 weakly suppresses
- L0:1224 weakly suppresses
- L13:132 weakly excites "of" after a noun
- L16:374 weakly excites
Feeds later layers
- L19:715 excites
- L21:2802 excites 'of' inside set phrases like 'course of'
- L18:2808 excites
- L20:64 suppresses
- L23:567 suppresses
- L20:2698 excites
- L22:160 excites
- L20:1972 excites 'of' in 'of course'
- L22:229 excites "of" after a noun in "X of Y"
- L25:286 excites
- L18:953 suppresses
- L19:1992 excites
- L20:2599 suppresses
- L21:2924 suppresses
- L25:2020 suppresses
- L23:128 excites 'for' as in 'for example'
- L18:2244 excites "of" inside titles and names
- L22:1993 suppresses
- L19:2507 suppresses
- L20:283 excites
- L22:2587 weakly excites
- L19:2088 weakly suppresses
- L22:288 weakly suppresses
- L18:2476 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- sorts
- appropri
- aths
- ự
- variables
- inclusion
- importance
- SCE
- circuits
- contr
Suppresses
- licate
- orial
- OfWork
- iotic
- →of
- eful
- liness
- headed
- aceous
- amework