'sh' starting a word piece
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 26 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the 'sh' sound at the start of a split word, as in 'shatter', 'shins', 'shelve' and 'shoveling'. When it was pushed up, the model fell into repeated symbols such as asterisks or line breaks, and when pushed down it produced rows of blanks.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
from deep pumped bores in vertical transmissive fault or shatter zones could augment mountain rivers for power generation. Flows might be augmented in
-
Implicit and Explicit Processing⏎We often learn a task by telling ourselves what to do. In the squat, I think about knees tracking out, butt going back, shins perpendicular, and opening up the hips on the way down. The explicit directions
-
to struggle to make reasons for not doing things they obviously ought to do. When the particular thing they ought to do involves the encouragement of art, their struggles to shelve the whole matter become quite painful. As an illustration of this, the case
-
million tons of plastic bottles. Plastic water bottles offer many advantages to both the manufacturer and the consumer. It is lightweight, which lowers transportation costs. It does not shatter, making it a safer option. However,
-
tibia) hurts when you press on it firmly with your finger.⏎The causes of shins splints include problems with body mechanics, a sudden or excessive increase in the
-
bi-products such as saw dust and planer shavings. This production demanded large amounts of water, and thus the pump tower and
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
and regulate blood pressure.⏎- Hijiki has more calcium than any other sea vegetable (14 times more calcium than milk), and is high in fiber.⏎- Nori contains as much protein as soybeans and
-
Описание презентации Презентация london-history-12162149479
-
flows, dotted: scoria cones. // in Elliot Hildner / University of Bremen⏎The main crack<|endoftext|>analysis of algoritms⏎robert.kern at gmail.com⏎Fri Sep 10 00:10
-
an ancient temple for Lord Krishna. Dwapara Yug<|endoftext|>Simple technique appears to be safe and effective, review suggests⏎By Robert Preidt⏎MONDAY, Oct. 15 (HealthDay News)
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L25:255 strongly excites pieces of split scientific or place names
- L19:1755 suppresses
- L14:2195 weakly suppresses
- L16:41 weakly suppresses
- L2:67 weakly suppresses
- L22:591 weakly suppresses
- L14:67 weakly suppresses
- L5:204 weakly suppresses
- L0:117 weakly suppresses
- L16:2438 weakly excites
- L5:168 weakly excites
- L24:244 weakly suppresses
- L22:229 weakly suppresses "of" after a noun in "X of Y"
- L12:467 weakly suppresses
- L15:1691 weakly excites
- L22:2398 weakly excites
- L18:1470 weakly suppresses comma after an introductory phrase
- L21:81 weakly excites
- L15:1116 weakly suppresses
- L0:1064 weakly suppresses
- L20:2102 weakly excites
- L24:331 weakly suppresses
- L21:881 weakly suppresses comma after 'if' clause in prose, braces in code
- L22:379 weakly excites
Feeds later layers
- L27:799 excites
- L27:860 weakly excites
- L27:136 weakly excites
- L27:445 weakly excites
- L27:2927 weakly excites
- L27:1466 weakly excites
- L27:519 weakly suppresses
- L27:1595 weakly excites
- L27:1630 weakly excites
- L27:1902 weakly excites
- L27:1472 weakly excites
- L27:2893 weakly suppresses
- L27:1053 weakly excites
- L27:207 weakly excites
- L27:2449 weakly excites
- L27:1535 weakly excites
- L27:1768 weakly suppresses
- L27:1326 weakly excites
- L27:2720 weakly excites
- L27:1527 weakly excites
- L27:1739 weakly excites
- L27:2688 weakly excites
- L27:1065 weakly excites
- L27:604 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- $⏎⏎
- …”⏎⏎
- ”⏎⏎
- $⏎⏎
- ”⏎⏎
- *⏎⏎
- »⏎⏎
- (){}⏎⏎
- !”⏎⏎
- ?”⏎⏎
Suppresses
- __
- остоя
- 到了
- 稠
- Recall
- ost
- oner
- 许
- FormData
- oods