'of' in set phrases like 'of course'
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 15 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
Fires on the word 'of' in phrases like 'of course', 'of all' or 'of interest'. It goes quiet or negative on 'of' after a number or quantity, such as 'one of' or 'a dozen of'. Turning it up made the model's continuations wander without a clear theme.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
and the telescope) can see. Those objects are getting farther away all the time because the universe is expanding, and moreover the expansion is accelerating. Strangest of all, the luminous matter of the galaxies—everything we see shining in the
-
a floral shop.⏎Even more influential of Liberia<|endoftext|>Oxford University Press⏎2017 | 200
-
was spoken around 3500 B.C.E. by a group of nomads called the Aryans living in what is now Southern Russia. It is of course critical to mention the improper usage of the word "Aryan" in N
-
primary process, few stop to ask a very important question: why do we have primaries in the first place? In the history of American democracy, political primaries are of relatively recent origin, so when and why did we adopt them?⏎Before answering these
-
use some, or all, of this content as you see fit, whether that’s to compliment, vary or boost work set by your child's school or, of course, just to give some extra structure to your usual EdPlace approach.⏎How
-
seems to be unique in the Indo<|endoftext|>A POTENTIALLY deadly and virulent bug that attacks the gut may soon be treated with the most stomach-churning of remedies.⏎A study published on Thursday in The New England Journal of Medicine has found
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the day the 2010, 2010, 2 the year, 2020, 2010, 20 10, 2 1 2 2
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
in the Federal Register, Rumsfeld decided to take things in hand. He assembled Jack Grayson, the chairman of the Price Commission, and about a dozen of our CLC staff and said that we wouldn’t be leaving until we had the
-
uranium. Heavy water – because it contains an isotope of hydrogen that has already absorbed an extra neutron and has no real affinity for absorbing another one – and one of several different gases with a low neutron affinity and a low molecular density.⏎Heavy water
-
’sot’ to the Quileute, referring to a place where there is a riffle in the water. Mora is located a mile from the mouth of the Quillayute River at its confluence with the Dickey River or
-
mealy bugs are a sign of overwatering.⏎A moisture meter is a great tool to help you keep track of your plant’s watering routine. Using one of these is a lot less hassle than trying to remember to water your succulents
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L1:725 suppresses "to" in "referred to"
- L3:315 excites 'of' after a noun phrase
- L0:2726 excites
- L6:357 weakly suppresses
- L4:4 weakly suppresses
- L2:1710 weakly suppresses
- L14:2417 weakly suppresses
- L0:40 weakly suppresses 'of' in 'sort of' hedges
- L5:363 weakly excites
- L3:294 weakly excites
- L14:435 weakly excites
- L3:1138 weakly excites
- L6:1566 weakly excites
- L11:1393 weakly excites
- L0:707 weakly excites
- L5:84 weakly suppresses
- L4:2080 weakly suppresses
- L13:2895 weakly excites
- L9:559 weakly excites
- L5:290 weakly excites
- L5:158 weakly suppresses
- L5:684 weakly excites
- L5:734 weakly excites
- L7:2916 weakly excites
Feeds later layers
- L17:317 suppresses 'of' after a noun phrase
- L22:229 suppresses "of" after a noun in "X of Y"
- L20:1972 suppresses 'of' in 'of course'
- L16:1088 suppresses 'of' after a noun phrase, as in 'of course'
- L16:2612 suppresses
- L19:715 suppresses
- L16:2412 suppresses
- L19:272 weakly excites
- L20:2698 weakly suppresses
- L26:2556 weakly suppresses
- L17:359 weakly suppresses
- L16:286 weakly excites
- L21:226 weakly excites
- L17:609 weakly excites
- L18:2808 weakly suppresses
- L16:817 weakly excites
- L25:286 weakly suppresses
- L18:277 weakly excites
- L16:1432 weakly excites
- L16:1344 weakly excites
- L19:2716 weakly excites
- L16:1662 weakly excites
- L17:264 weakly excites
- L17:253 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- سطين
- ・
- ện
- Throws
- ottom
- ival
- 一面
- rances
- 之
- ственно
Suppresses
- 经开
- bral
- chá
- ","+
- =',
- ApiKey
- =admin
- mys
- ذي
- >[]