end-of-document token
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 9 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron lights up on the hidden marker that separates one web page or document from the next. Turning it up mostly produced rambling, unrelated text, so it does not seem to shape any clear topic.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
Chapter 6 Exercise 6.5, we have discussed the Pythagoras theorem. Other important topics related to the chapter are covered in the following exercises.<|endoftext|>What you will learn in this module⏎Understand different tasks that have to be
-
two different numbers cant have the same cube). Therefore, by Definition 1, is one-to-one....⏎View Full Document⏎- Spring '09<|endoftext|>part of speech: noun⏎definition 1: an upright line or pole with
-
allowed us to propose something that hasn´t yet been possible to study in depth and that now should be taken into serious consideration. Is genomic instability the cause of tum<|endoftext|>The role of the built environment in addressing climate change is clear as buildings account for
-
(e) William Sholes - Typewriter⏎(f) Louis Daguerre - Photography⏎- (a) Lewis Waterman⏎(b) Lazlo Biro<|endoftext|>We are
-
ions iv. Special Limits c. Will be seen later d. If none of the top three apply, try using the Squeeze Theorem....⏎View Full Document<|endoftext|>A Woman of Salt, by Mary Potter Engel⏎Midrash refers to the
-
can be claimed, the final question is what, exactly, has been discovered? Scientists can have complete confidence in their results but remain uncertain about how to interpret them<|endoftext|>After
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: a new program, which is a set of rules that you can use to talk about it again and again and again and again and again and again and again and again and again and again and again and again and again and again and again and
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
at high risk for developing osteoporosis. Osteoporosis is a condition in which thinning bones become so fragile that they are prone to fracture easily. A person with osteopenia may benefit from treatment aimed
-
, and sometimes they also tell us about what it was like when it was young. That’s due to gravitational lensing, a peculiar effect of relativity where the overall gravity of the cluster bends the light of an even more distant galaxy behind it
-
you may be entitled to a settlement.⏎The most common type of head injury is a concussion or traumatic brain injury (TBI) and can happen when the body is impacted with enough force to shift the brain and it collides with the skull.
-
financial literacy education for grades K-12, college, and post-college.⏎To view this DRM protected ebook on your desktop or laptop you will need to have Adobe Digital Editions installed. It is a free software. We also strongly recommend
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L0:783 excites end-of-document marker token
- L6:3068 weakly excites
- L2:2888 weakly suppresses
- L5:236 weakly suppresses
- L6:994 weakly suppresses
- L7:2309 weakly suppresses
- L4:158 weakly suppresses
- L5:294 weakly suppresses
- L7:792 weakly excites
- L4:790 weakly suppresses
- L0:436 weakly suppresses
- L7:383 weakly suppresses
- L7:2383 weakly suppresses
- L3:186 weakly suppresses
- L6:67 weakly excites
- L7:1333 weakly excites
- L8:2757 weakly excites
- L2:1141 weakly suppresses
- L7:1423 weakly suppresses
- L2:222 weakly excites
- L7:1355 weakly suppresses
- L6:505 weakly suppresses
- L4:1963 weakly suppresses
- L0:1937 weakly suppresses
Feeds later layers
- L17:712 excites
- L17:2097 weakly suppresses
- L12:1003 weakly suppresses
- L10:1461 weakly suppresses
- L19:1732 weakly suppresses
- L19:1698 weakly suppresses
- L22:389 weakly suppresses
- L25:811 weakly suppresses
- L24:967 weakly suppresses
- L18:2322 weakly excites end-of-text token after a finished passage
- L16:588 weakly suppresses
- L23:388 weakly suppresses
- L27:42 weakly excites
- L24:1919 weakly suppresses
- L11:3059 weakly suppresses
- L17:1402 weakly suppresses
- L16:2324 weakly suppresses
- L12:2078 weakly excites end of document token
- L27:825 weakly suppresses
- L17:748 weakly suppresses
- L11:1123 weakly suppresses
- L24:1849 weakly suppresses
- L17:1105 weakly suppresses
- L25:2795 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- iel
- et
- el
- ens
- arian
- innacle
- !
- esar
- uel
- lector
Suppresses
- же
- _)
- 电子邮件
- teness
- activex
- damer
- 又要
- ***/⏎
- 我们要
- @return