slot right before a quantity
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 8 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron becomes active on the empty spot just before a number, such as "the other 47 states" or "losing just 5 to 10 percent." It seems to expect a count or amount to come next. Its negative side appears on words like "fifth" and "percent." Steering changed the topic toward maths and numbers, but not consistently.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
6 percent). Only one state, New Hampshire, had a poverty rate of less than ten percent for women, at 8.9 percent. In the other 47 states and the District of Columbia, between 10 and 2
-
60 second-, third-, and fourth-grade students, while 48 second- and fourth-grade students were subjects in the other 3 studies. Results indicated that: (1) younger children tended to have difficulty
-
warming has already reduced rainfall and increased evaporation in the Mideast, worsening water shortages. Up to now, climate scientists had projected that rainfall could decline another 20 percent by 2100. But the Dead Sea cores suggest
-
ics, for another 75,000 to 1.75 million years. After
-
at increased risk for type 2 diabetes, heart disease, osteoarthritis, and certain forms of cancer.⏎âIf youâre overweight, losing just 5 to 10 percent of your body weight can slash the risk of both
-
counting 1, 2, 3, 4 like this:⏎The first note is held for 4 counts, the second bar each note is held for 2 counts and, in the
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the impact of a natural disaster on the results of the calculation. So, what are the effects of a natural disaster on the results of the calculation? I have been trying to find a way to find the effect of a natural disaster on
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
greatest challenges.⏎Roughly 1.7 million children under five years old die every year in India, says Unicef, representing more than a fifth of all child deaths worldwide. Most of these deaths are preventable as they stem
-
, the Greeting Card Association notes that over 1 billion valentines are sent annually, vs. 2.5 billion Christmas cards; 85 percent of all valentines are purchased by women.⏎Vintage Valentines are increasingly popular
-
—makes up less than one-half of 1 percent of what’s out there. Most of the universe is unseen
-
are unsafe within the confines of the military system. According to the Department of Defense, almost a quarter of all women in the military experience sexual harassment, and more than 20,000 soldiers were sexually assaulted in the 20
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L7:1742 suppresses
- L7:1524 suppresses
- L6:1469 weakly excites
- L1:1095 weakly excites
- L6:2006 weakly excites
- L7:224 weakly suppresses
- L1:1802 weakly suppresses
- L3:2177 weakly suppresses
- L7:1388 weakly excites
- L6:1562 weakly excites
- L7:1114 weakly suppresses
- L7:583 weakly suppresses
- L6:457 weakly excites
- L5:276 weakly suppresses
- L4:1555 weakly suppresses
- L6:345 weakly excites
- L5:2724 weakly suppresses
- L7:531 weakly suppresses
- L1:1847 weakly suppresses
- L3:293 weakly excites
- L5:2029 weakly excites
- L7:2496 weakly excites
- L1:2281 weakly suppresses
- L7:398 weakly excites
Feeds later layers
- L17:222 excites
- L17:1570 excites
- L9:97 suppresses
- L16:1897 excites
- L11:579 excites
- L14:471 excites
- L12:1467 excites numbers after dates and figures
- L9:335 suppresses 'of' after nouns like anniversary or onset
- L10:2063 excites
- L10:868 excites
- L16:399 excites
- L12:408 suppresses
- L25:1212 excites
- L13:2915 excites
- L13:855 excites
- L18:1259 excites
- L19:1419 excites
- L13:1308 excites
- L15:2091 excites
- L13:122 suppresses
- L21:676 excites
- L25:46 suppresses
- L11:751 excites numbers and quantities mid-phrase
- L19:1117 excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- 三千
- nero
- 五千
- eron
- significant
- 几百
- 几千
- 众多
- banyak
- semiclass
Suppresses
- jec
- äng
- etten
- sujet
- нашего
- uck
- ận
- ться
- uve
- agle