the blank spot just before a number
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 22 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the empty space right before a number, such as the gap after 'up to' or 'about' when a quantity comes next. Its steering results are messy and only loosely involve numbers.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
Colin Johnston in his paper on the “Science of Sunshine,” the time it takes photons to get from the core of the Sun to its surface could take up to 40,000 years. This is because the photons have to travel
-
irical fiction, Knickerbocker’s History of New York, with numerous references to<|endoftext|>county in the W. part of Vermont, has an area of about 750 square
-
, and even sniping ability. Its combination of exceptional mobility – being light enough to be carried by one man – and high rate of fire (of up to 900 rounds per minute) was unmatched. It entered service in great numbers
-
, which will continue to grow with NASA’s latest photos of Mars. The images below cover Curiosity’s historic landing and NASA’s following press briefing.⏎On August 5, 2012 at 10:31 p.m
-
of Pakistan borders Afghanistan, India, Iran. Gross domestic product (GDP) is of about 315000 millions of dollars dollars.⏎K2 is the highest
-
at the Ingleside Response Center at 361-396-2831. Or if you
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: what if speaking care�建宾-harbar-harbar-harbar-harbar-harbar-harbar-harbar-harbar-harbar-harbar-harbar-harbar-harbar-harbar
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
between ourselves and God.)⏎2. Can you pray when you are in trouble? (Yes.)⏎3. Can you pray when you are happy? (Yes.)⏎4. What is another name for happy prayers? (Praises.)⏎5. What
-
00 on a ship carrying 2200). There was the steaming ahead at full-speed despite various warnings about the ice-field. There was the lack of binoculars for the lookout. There were the poor procedures with the new
-
they handle their schoolwork or chores around the house. If they are reliable at age 13 and listen to you (as well as a 13-year-old can
-
2. Explain the growth of a hardened conscience.⏎3. What is the relation of external action to love? What is the "heresy of action"?⏎4. What is the relation of love to emotion?⏎5. How can one use
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L20:326 strongly suppresses
- L19:440 strongly suppresses
- L21:427 strongly excites
- L10:80 strongly suppresses
- L20:70 excites
- L19:753 excites
- L15:66 suppresses
- L20:329 excites
- L15:562 suppresses
- L16:2610 excites
- L18:332 excites
- L14:1266 suppresses
- L18:1633 suppresses
- L15:2706 excites
- L18:664 excites
- L18:232 excites
- L10:58 suppresses
- L18:1115 excites
- L20:294 suppresses
- L16:926 suppresses
- L17:283 suppresses
- L3:157 suppresses
- L18:364 excites
- L15:1426 excites
Feeds later layers
- L26:732 strongly suppresses
- L25:409 strongly suppresses
- L26:315 suppresses
- L24:363 suppresses
- L26:134 excites
- L25:148 suppresses
- L26:60 suppresses
- L27:78 excites
- L25:5 excites
- L27:750 weakly suppresses
- L24:226 weakly suppresses
- L26:547 weakly excites
- L27:454 weakly suppresses
- L26:885 weakly suppresses
- L26:37 weakly suppresses
- L25:420 weakly suppresses
- L27:581 weakly suppresses
- L27:506 weakly suppresses
- L25:2375 weakly suppresses
- L26:5 weakly excites
- L26:20 weakly excites
- L27:26 weakly suppresses
- L27:404 weakly excites
- L26:159 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- '
- «
- 『
- {'
- "
- «
- ‘
- ‘
- local
- at
Suppresses
- ghan
- aldi
- -bar
- óst
- -bars
- iero
- enha
- ibur
- hta
- yssey