the spot before a number after 'less than' or 'more than'
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 5 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
The neuron fires on the spot just before a number in phrases like "less than", "more than", "over" or "about", where a quantity is about to follow. Turning it up produced text about numbers and counting, though it was not consistent.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
30 U.S. agencies involved themselves in war relief. The ARC, the oldest and most organized relief group in 1914, had less than 150 chapters and twenty thousand members. The agency had a new national headquarters
-
effusion. It is also done to relieve the shortness of breath that a large pleural effusion can cause.⏎Normally the pleural cavity contains less than 20 milliliters (4 teaspoons) of clear, yellowish (
-
tight squeeze through columns and a near-siphon to a low, broad breakdown room. A tunnel to the north drops 8 feet into a trench containing from 3 to 5 feet of water. Several side passages branch off from this section
-
Desert can be divided into three main geographic regions depending on their landscape: sand, plains and hills. There are also lakes and salt marshes. With less than 10 inches of rainfall in 2016, the temperature in this
-
single migratory event. If an earlier and ultimately failed voyage occurred, it left little trace in the genomes of people alive today.⏎The human genome contains more than 20,000 protein-coding genes, which carry the instructions for
-
in California, which represent more than 700,000 people. The Karuk is the second largest Native American tribe in California and has over 3,200 members. Their tribal lands include over 1.4
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: This, I have been thinking about the (1) in the morning. So, this is the, and so So I The So I So I So I So I So
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
chakras all over our bodies; they are everywhere in and around us. They are all over the planet at sacred power points on the land. They are everywhere within and without.
-
two tribute sites Faces from the Lewis War Memorial and Second World War Tribute for Lewis.⏎The full list of war memorials in Lewis:<|endoftext|>I’ve seen my fair share of horrible PowerPoint presentations, and I’ve noticed that they tend to follow common
-
sstatement: said, “This is the inspiration for this year’s theme, Gender equality benefits everyone. It is a powerful reminder that we all have so much to gain when we advance gender equality. It is both the right thing to do and
-
once told Jacob (Gen. 26:4-5), “I will make your seed to multiply as the stars of heaven, and I will give to your seed all these countries. And in your seed shall all the nations of the
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L3:1218 suppresses
- L3:293 weakly excites
- L4:2702 weakly excites
- L3:59 weakly excites
- L3:489 weakly suppresses
- L3:465 weakly suppresses
- L2:188 weakly excites
- L1:2006 weakly suppresses
- L0:3015 weakly excites
- L4:234 weakly excites
- L0:689 weakly suppresses
- L1:1152 weakly suppresses
- L3:753 weakly suppresses
- L0:2446 weakly suppresses
- L3:863 weakly suppresses
- L2:2607 weakly suppresses
- L0:311 weakly excites
- L4:684 weakly suppresses
- L2:302 weakly excites
- L4:1788 weakly excites
- L4:2562 weakly suppresses
- L3:2791 weakly excites
- L0:1187 weakly excites
- L3:1407 weakly suppresses
Feeds later layers
- L8:1029 excites
- L11:2346 excites
- L11:979 excites
- L20:1404 excites
- L10:141 suppresses
- L16:2448 excites
- L23:1404 excites
- L22:825 excites 'for' before a span of time
- L19:2851 excites
- L18:1006 weakly excites
- L10:960 weakly excites
- L17:2587 weakly excites
- L16:1154 weakly excites
- L17:2447 weakly excites
- L10:2300 weakly excites
- L19:701 weakly excites
- L16:2055 weakly excites
- L6:1697 weakly excites
- L19:1528 weakly excites
- L8:1368 weakly excites
- L12:2953 weakly suppresses
- L6:625 weakly excites
- L8:199 weakly suppresses
- L6:1582 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- оличество
- 만큼
- 足够的
- 足够
- imately
- enough
- ano
- anio
- aven
- casualties
Suppresses
- :border
- 说我
- 昧
- 我发现
- ::⏎
- ()?;⏎
- 还以为
- (environment
- 就知道
- 当我