line breaks inside wrapped sentences
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 5 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
The neuron fires on the invisible line break that a text has partway through a sentence, where the line was wrapped by a page or column rather than ending a thought. Turning it up did not make the writing look or read differently in any clear way.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
-splicing and in some cases can act⏎as an enzyme. A wide variety of rRNA molecules and some mRNA molecules (chloroplast and mitochondrial) can splice⏎themselves. These molecules are called ribozymes (RNA enzymes; Figure
-
areas mark⏎seismically active regions in the channel between Fogo and Brava (seismic crisis 1994) and at Cadamosto⏎seamount (Heleno et al 1999; Grev
-
the double⏎helix is said to "melt" into its two component molecules, and will⏎reassociate upon slow cooling under appropriate salt conditions. So,⏎the issue is not decomposition so much as disassociation upon exposure⏎to too high
-
to specify, visualise,⏎construct and document the artifacts of an object-oriented software-intensive⏎system under development. The UML represents a compilation of "best engineering⏎practices" which have proven successful in modelling large, complex systems.⏎UML
-
race for a cure holds a promise for many struggling with the disease.⏎Eight million people die from cancer worldwide every year, out of which four million people die prematurely⏎(aged 30 to 69 years). Leading cancer risk factors include
-
works. Normally when you try and cd⏎into a directory, it will only try and change to the path specified from the current directory. If you spend a⏎lot of time working in specific folders, it might be useful to assume that your
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: "I know, I'm not going to be there, I'm just going to be there, I'm just going to be there...". So, the "T" problem, and the "T" problem... It's been a
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
that the foe, or their progenitors,⏎had been citizens of Colordo in the far West. And that,⏎having conquered all before them, they sought another⏎World to conquer here.⏎When we do trace out nature's laws,⏎And view
-
bars,⏎When, oh! the welcome news then came,⏎To get abound the cars.⏎They marched us through their nasty streets,⏎That all the crowd might see,⏎The “hiding Yankees” from the North,⏎That in than land run free
-
two or more servers do not interfere⏎with each other. Usually NFS file locking is handled by the⏎lockd daemon, but at the moment there is no⏎platform that performs locking 100% reliably in every situation.⏎With this
-
) Tetragonal System :⏎Three axes make right⏎angles with each other. The two horizontal⏎axes, a and b, are of equal length but the⏎third, c axis, is of different length, e.g⏎(iii)
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L4:863 suppresses
- L4:351 suppresses
- L3:42 excites
- L3:428 excites
- L3:1592 suppresses
- L4:114 excites
- L4:1230 weakly suppresses
- L4:19 weakly suppresses
- L4:71 weakly suppresses
- L0:1235 weakly suppresses
- L2:2692 weakly excites
- L1:243 weakly suppresses
- L4:3027 weakly excites
- L3:1597 weakly excites
- L2:2740 weakly suppresses 'on' before time or track
- L2:477 weakly excites line break in the middle of a sentence
- L1:2612 weakly excites
- L3:1355 weakly suppresses
- L1:1298 weakly suppresses
- L3:421 weakly suppresses
- L1:557 weakly excites
- L2:44 weakly excites
- L4:198 weakly suppresses
- L3:1752 weakly excites
Feeds later layers
- L6:754 excites
- L7:1240 suppresses
- L7:239 excites
- L6:710 suppresses
- L9:234 weakly excites line break inside hard-wrapped text
- L19:771 weakly suppresses
- L13:2286 weakly suppresses
- L11:1272 weakly suppresses unclear: sentence ends in factual prose
- L22:2535 weakly suppresses
- L14:3000 weakly suppresses
- L24:574 weakly excites
- L6:574 weakly excites
- L6:1060 weakly excites
- L11:423 weakly suppresses
- L14:2237 weakly suppresses
- L7:1539 weakly excites
- L20:564 weakly excites
- L16:2235 weakly excites
- L8:2501 weakly suppresses 'iv' inside herbivores, energy fuel words
- L23:825 weakly excites
- L22:2576 weakly excites
- L7:1360 weakly excites
- L21:1557 weakly suppresses
- L21:2827 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- ź
- مخ
- vido
- 集聚
- bage
- ķ
- ţi
- owi
- ORIES
- -package
Suppresses
- endforeach
- essed
- iii
- /environment
- III
- .Arguments
- 格林
- .cloudflare
- ellas
- iii