blank space before a number
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 24 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the empty space in a phrase like 'a [space] 64-bit' or 'a [space] 35-millimeter', where a number is about to follow. Turning it up made the model write long lists of digits, which fits the same idea.
Written by Claude from this neuron's activations and steering outputs. Confidence: high. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
the garden bed by using a garden fork or tiller to loosen the soil to a depth of 12 to 15 inches, then mix in a 2- to 4-inch layer of compost and/or aged manure.⏎Plant
-
the Gemini IV mission during orbit no. 19, on June 4, 1965, with a Hasselblad camera and a 70mm lens. The Gemini IV crew — astronauts Jim McDivitt and
-
kinds of combinational circuits are simulated and compared with RALA's. RALA shows the best performance in delay time, area, and power product results. A 64-bit carry-look-ahead adder with RALA is fabricated by
-
conducted a study on the movement patterns of round gobies in Muskegon Lake. Using PIT tagged fish we followed 48 round gobies enclosed in a 20x20
-
-2 rocket lifted the first ever space-bound camera above New Mexico on a short sub-orbital flight. From an altitude of 65 miles, a 35-millimeter motion picture camera captured grainy monochromatic views of
-
depending on where you are. I set up my camera set my lens to its lowest aperture (for my 18mm kit lens it is f/3.5) and then focused
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the early early ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
along a 100-kilometer long north-south linear vent system over the past 5 million years;
-
on the environment as 710,814 flights from London to New York, or eating 16,375,055 kilos of beef.⏎Research shows “a sustained loss of 3 ± 0
-
In other words, this is the income that one person needs to meet the cost of basic needs which include the cost of food to provide 2030 kilo calories per person per month, clothing, housing, education, transport, and
-
of 15,795 razorback sucker and 11,696 bonytail were released with 134.2 kilohertz (kHz) PIT tags into the Colorado River downstream of Palo Verde Div
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L23:2142 suppresses
- L22:2490 excites
- L22:2871 suppresses
- L22:2645 suppresses
- L18:291 suppresses
- L5:222 suppresses
- L18:797 excites
- L23:2601 suppresses unclear: numbers and distance words
- L21:326 suppresses
- L17:2446 weakly suppresses
- L21:1203 weakly suppresses
- L20:2436 weakly suppresses
- L22:1574 weakly suppresses
- L17:2827 weakly excites
- L13:1736 weakly suppresses
- L16:1919 weakly suppresses
- L22:481 weakly excites
- L21:644 weakly suppresses
- L21:2146 weakly excites blank before a size number
- L14:102 weakly suppresses
- L22:2775 weakly excites
- L6:1626 weakly suppresses
- L23:2791 weakly suppresses
- L4:1216 weakly excites
Feeds later layers
- L25:253 strongly excites
- L26:667 strongly excites
- L27:287 strongly excites
- L25:46 strongly suppresses
- L27:113 strongly suppresses
- L26:476 strongly excites
- L27:2172 strongly suppresses
- L26:2953 strongly suppresses
- L27:978 excites
- L25:2902 excites
- L27:163 suppresses
- L26:172 excites
- L26:423 suppresses
- L26:329 suppresses
- L26:611 excites
- L25:854 excites
- L27:1456 excites
- L27:428 excites
- L26:991 excites
- L27:962 excites
- L27:448 suppresses
- L27:2089 excites
- L26:1127 suppresses
- L27:1132 suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- 1
- thirty
- fifty
- 1
- 3
- 3
- 5
- seventy
- twelve
Suppresses
- ipples
- cedes
- //**⏎
- nicas
- ANDOM
- IGNAL
- nds
- adow
- .uf
- pired