gap before a second number
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 11 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the empty spot right before a second number in a pair or range, such as the "12" in "7,12" or the "1903" after "and". Turning it up gave replies that did not clearly relate to numbers, so the steering was not informative.
Written by Claude from this neuron's activations and steering outputs. Confidence: low. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
:3-6)⏎- How did Herod scheme against Jesus? How did God protect Jesus from Herod? (Matthew 2:7-8, 12)⏎- Where did the wise men find the star? (Matthew
-
States national anthem, "The Star Spangled Banner"—but was commonly known as 'Scott'. He spent 1898–1901 and 1903–1908 in Buffalo, New York, where
-
as Demonstrated by Rubidium-Marked Eggs," American Journal of Tropical Medicine Hygiene⏎52 (2):177-179, 1995).⏎View additional Information on Aedes Aegypti
-
zua was described by Marcus (1955) from the coast of Sao Paulo, Brazil. 3 specimens in June 1953, 20 eating Codium in April 1954. Intertidal
-
adults get at least 2 1/2 hours of moderate-intensity cardio exercise each week. Alternately, you can perform an hour and 15 minutes of vigorous cardio exercise. By regularly performing cardio exercise, you can
-
of the plate boundary types, which type of boundary has the most earthquakes on this list?⏎Put the settings on 7 days, 4.5+ worldwide. Find a region where there is a high concentration of
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the number of 3D objects. I think about the number of points and objects. Maybe I have a lot of points or 1000,000 objects, and I'm thinking about 3D objects. I
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
3(3), pages 315-320.⏎- Angus Deaton & Olivier Dupriez, 2
-
lorescence: raceme; bracts leaf-like⏎Flower generally nodding, bell- or cup-shaped; perianth segments 6, of 2 similar whor
-
after birth.⏎Pig clones have popped up in the United States, the United Kingdom, Japan and other locations,<|endoftext|>Cleopatra, queen of Egypt and lover of Julius Caesar and Mark Antony, takes her life following the defeat of her
-
not able to check, treat for low blood sugar anyway.⏎Give your child 15 to 20 grams of fast-acting sugar such as 3 to 4 glucose tablets, a glass of nonfat milk, or 4 ounces
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L7:25 suppresses
- L6:2945 suppresses
- L8:70 excites 'and' between stacked superlatives
- L9:1449 suppresses
- L6:1084 suppresses
- L6:427 suppresses
- L6:1706 suppresses
- L9:2010 suppresses
- L9:212 excites
- L6:172 excites
- L10:2944 excites
- L7:576 suppresses
- L8:593 suppresses
- L10:2046 suppresses
- L9:229 excites
- L9:92 suppresses "and" or "or" joining two items
- L10:1171 excites
- L9:2726 excites
- L9:1374 excites
- L8:57 suppresses
- L4:57 suppresses
- L3:250 excites
- L9:2516 excites
- L8:1608 excites
Feeds later layers
- L12:1212 excites
- L12:1197 excites
- L12:959 excites
- L14:844 excites
- L12:123 suppresses
- L15:38 suppresses
- L16:492 suppresses
- L16:1294 excites
- L14:982 excites
- L12:1756 excites
- L12:710 excites
- L13:517 excites comma closing a list item
- L13:125 excites
- L15:217 suppresses
- L17:84 suppresses
- L15:986 excites
- L13:371 suppresses
- L21:735 excites 'or' in ranges and alternatives
- L13:537 suppresses
- L13:716 excites
- L14:745 excites
- L14:2054 excites
- L15:745 excites
- L15:1162 excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- thed
- bite
- etic
- naments
- 问他
- 벤
- vale
- ections
- HECK
- .getB
Suppresses
- .TypeString
- 扩大
- ymb
- 这点
- oli
- 这方面
- 这一点
- 开头
- Ini
- Reject