comma after a vocative or lead-in
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 20 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on a comma that comes right after a short lead-in, such as a name being addressed ("Why would I do that, Marcus?"), an exclamation ("Great work, Shipmates!") or a quoted speaker. Turned up, the model fell into repeating words like "my dear" and "Ms.". The negative side is mostly C code with "void" pointers.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
in My Garden”, beloved local garden guru Carolyn Singer wrote, “’Grandma, why don’t you grow yucky flowers?’ ‘Why would I do that, Marcus?’ I asked, respectful of his three-year-old wisdom. ‘Well,’
-
.⏎Resilience is his middle name.⏎Here circumstances are presented as clouds that help trouble and strife to germinate and proliferate.⏎‘This is the world, and we cannot expect to give many hours to pleasure – Circumstances are like
-
, as a way of depicting his self-grandeur. In one of her books, another character asks him about it:⏎- Dr. Lutz: Tell me, why do you insist on referring to yourself in the third person? It's intensely
-
start through direct mail solicitations. "The shriller you are, the better it is to raise money," explained co-founder Terry Dolan.⏎N
-
said Guillaume Gronoff, a French astrophysicist at NASA Langley.⏎"And you think, 'OK, what I am looking at — it's a<|endoftext|>An ultra-
-
makes you wonder why Moses included it where he did, or at all. And even more, "Why, now is this seemingly obscure story from the Book of Genesis not only included in the
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: what is the value of the value of the value of the value of the value of the value of the value of the value of the value of the value of the value of the value of the value of the value of the value of the
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
conuts and banana leaves<|endoftext|>const void * memchr ( const void * ptr, int value, size_t num );⏎void * memchr ( void * ptr, int value, size_t num );⏎Locate character in block of
-
same to a compiler:⏎As humans though, the above is a lot more like gibberish.⏎void * load_model_from_disk(const char *);⏎void draw_model(void *, float, float, float);⏎void deallocate_model(void *);⏎And that is
-
ptr.⏎If the value is not found, the function returns a null pointer.⏎In C, this function is only declared as:⏎void * memchr ( const void *, int, size_t );⏎instead of the two overloaded versions provided in C++
-
is recorded in several Roman mosaics.⏎The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept"
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L16:2930 suppresses
- L18:1111 excites
- L17:2715 weakly suppresses
- L19:199 weakly excites
- L18:569 weakly excites
- L19:650 weakly suppresses
- L16:1854 weakly suppresses
- L7:62 weakly excites
- L19:1662 weakly suppresses
- L19:1386 weakly excites
- L14:2640 weakly suppresses
- L5:158 weakly suppresses
- L10:1773 weakly excites
- L5:60 weakly excites
- L2:2083 weakly excites
- L16:1834 weakly excites
- L1:1487 weakly suppresses
- L15:1122 weakly excites
- L2:1583 weakly suppresses
- L7:56 weakly suppresses
- L5:110 weakly suppresses
- L9:846 weakly excites
- L19:2672 weakly suppresses
- L8:7 weakly excites
Feeds later layers
- L22:399 excites
- L23:436 excites
- L21:772 excites
- L25:383 excites
- L23:2835 excites
- L24:340 excites
- L25:343 excites
- L22:3063 excites
- L21:2385 excites
- L27:634 excites
- L21:2834 excites
- L27:412 weakly excites
- L24:1658 weakly excites
- L26:970 weakly excites
- L22:1296 weakly excites
- L23:386 weakly excites
- L21:2053 weakly excites
- L22:324 weakly excites
- L21:696 weakly excites
- L22:712 weakly suppresses
- L24:2040 weakly excites
- L23:24 weakly suppresses
- L21:306 weakly excites
- L22:368 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- 宝贝
- 朋友
- 亲爱的
- gentlemen
- friend
- 伙伴
- 小伙伴
- 同学
- buddy
- 儿子
Suppresses
- 把她
- yscale
- hers
- *self
- !)⏎
- arı
- .cuda
- *c
- Ticks
- .INTER