end-of-document marker token
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 0 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron reacts to the hidden end-of-text marker that separates one web page or document from the next. Turning it up made the model's writing wander into unrelated topics, with no clear theme.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
if not server_response['successful']:⏎etag = server_response['file_handle'].info().getheader('ETag')⏎xml_document = server_response['xml_document']<|endoftext|>Deepak Vashishth, assistant professor of biomedical engineering, hopes that his
-
одальные глаголы и их эквиваленты и выполните следующее задание:⏎а) заполните пропуски<|endoftext|>Last Updated February
-
glad it wasn't just us who saw the poor editing.”⏎‘golden-eye’ hasn't been added to any lists yet.⏎Looking for tweets for golden-eye.<|endoftext|>New subjects guide for primary years⏎Designated
-
Paper 294.⏎Loma Linda University Electronic Theses and Dissertations⏎Loma Linda University. Del E. Webb Memorial Library. University Archives<|endoftext|>Category Archives: History⏎There were eighteen African-American athletes
-
-12⏎Latest update: 2011-JUN-19⏎Author: B.A. Robinson⏎This page translator works on Firefox,<|endoftext|>It looks like you're using Internet Explorer 11 or older
-
Otago’s Department of Psychology, says “we may have to seriously re-think the use of the term ‘bird brain’ as a put down.”<|endoftext|>While doing research in a local newspaper for my ancestors in Harvard, Illinois, I
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: a new idea. Let me present this new idea to you. It is the idea that I have been thinking about lately. Let me present this new idea to you. It is the idea that I have been thinking about lately. Let me
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
Meng Haoran⏎Era Meng Haoran Major poet during the Tang Dynasty Setting of Spring Dawn⏎Meaning for Line 1 春 : spring 眠 : sleep 不 : not 觉 : to feel, wake up 晓
-
it in more detail, because it’s a really cool parasite, as well as in the center of much research nowadays.⏎Wolbachia are Gram-negative ⍺-Proteobacteria (order Rickettsiales, family Anapl
-
, 24 = 4 ⋅ 6. So to solve the second part, on average, we need
-
With time we check more numbers and become more convinced that the conjecture is true. Currently, it has been checked up to the power 4.6 ⋅<|endoftext|>Picture Yourself in Orbit: Are You Really Weightless?⏎An object in
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
None: this is the first layer.
Feeds later layers
- L9:303 excites end-of-document token
- L5:236 excites
- L6:407 excites end-of-text token after a web page
- L26:1 weakly suppresses
- L1:1592 weakly suppresses
- L26:2977 weakly suppresses
- L24:2340 weakly suppresses
- L24:2775 weakly excites
- L26:1449 weakly suppresses
- L13:1681 weakly suppresses
- L15:694 weakly excites
- L23:44 weakly excites
- L27:2105 weakly suppresses
- L18:627 weakly suppresses
- L4:41 weakly excites
- L24:224 weakly excites
- L17:2864 weakly suppresses
- L9:1641 weakly suppresses
- L1:3017 weakly suppresses
- L3:269 weakly excites
- L4:204 weakly suppresses
- L25:2540 weakly suppresses
- L6:2101 weakly excites
- L2:1771 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- ChildIndex
- ushi
- configuration
- .Profile
- 📢
- #${
- ATFORM
- ENTION
- .CONFIG
- Alphabet
Suppresses
- own
- iders
- vars
- own
- rates
- Rates
- чество
- idor
- orde
- edd