end of document marker
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 18 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron lights up when one piece of text ends and an unrelated one begins, at the invisible end-of-document marker. Turning it up made the model's writing break down into repeated words and loops.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
target values⏎removes empty columns⏎imputes missing values with mean values⏎To remove default preprocessing, connect an empty Preprocess widget to the learner.<|endoftext|>Start Date: September 2012⏎Robert Waymouth and Christopher Ch
-
for the search of both extant and fossil biomarkers, but sequential extraction would be required to optimize the analysis of each of these in turn.⏎// end //<|endoftext|>Michigan has a diverse animal population because of its prime location between two of the world
-
A value of one represents⏎the entire of the character bounding box from the bottom of any⏎possible descenders to the top of any possible ascenders.<|endoftext|>What is Red Data Book?⏎The Red Data Book is a public document which is
-
agus níl aon rochtain againn ar shonraí a bhaineann le do chárta creidmheasa.<|endoftext|>WASHINGTON — Ancient bison DNA may help explain a longstanding mystery of North American wildlife
-
oid(), take three parameters called width, length and height and apply the relevant formula to calculate and return the volume of a cuboid based on the given dimensions.<|endoftext|>Approximately 35% of the United States population suffers from Divertic
-
wellandgood.com.<|endoftext|>The Technology Behind Bella Hadid’s Spray on Dress⏎During the 20
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the idea of a specific, clearly defined, and successful, and by the end of this, I have found out about the details, and the other information, and the information about the other people, and the other information, and the other
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
100 it gets 1st grade and is very good.⏎If the sheep gets<|endoftext|>|Republic of Latvia⏎|Anthem: Dievs, svētī Latviju!⏎God Bless Latvia!⏎and largest city⏎|Eth
-
then else-syntax. However, since it is a function instead of syntax, it can be partially applied and passed around to higher order functions, like so:⏎ghci> :m + Data.Bool.Extras⏎ghci> let yesOrNo
-
be achieved with the following code:⏎TYPE List = REF RECORD next: List; value: INTEGER; END;⏎VAR x := NEW(List, value:=1); BEGIN x.next :=
-
Old Ind. grīva- “neck,” grīvā- “the back part of the
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L8:493 suppresses
- L17:834 excites
- L10:1270 excites
- L16:496 excites
- L15:574 suppresses
- L13:643 suppresses
- L3:1993 suppresses
- L4:1 excites
- L16:1295 excites
- L4:1605 suppresses
- L17:2603 excites
- L15:1048 excites
- L3:968 weakly excites
- L16:871 weakly suppresses
- L16:772 weakly suppresses
- L6:2467 weakly suppresses
- L8:407 weakly excites
- L9:1369 weakly suppresses
- L8:51 weakly excites
- L4:1428 weakly suppresses
- L3:1933 weakly excites
- L4:1173 weakly excites
- L15:524 weakly excites
- L3:5 weakly excites
Feeds later layers
- L22:1400 excites
- L21:341 excites
- L20:910 excites
- L20:843 excites
- L19:1650 excites
- L19:796 excites
- L22:2957 excites
- L20:2199 excites
- L20:3033 excites
- L20:508 suppresses
- L20:2899 excites
- L24:2050 excites
- L20:2816 excites
- L19:3011 excites
- L21:3045 excites
- L19:1280 excites end-of-document token between unrelated texts
- L22:2553 excites
- L22:490 excites
- L22:1105 weakly excites
- L19:1404 weakly excites
- L21:1548 weakly excites 'at' in time and occasion phrases
- L21:659 weakly excites end of document boundary
- L25:733 weakly excites
- L21:2243 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- BoxFit
- .lua
- !$
- aravel
- inston
- .scala
- NGX
- .dart
- еж
- üst
Suppresses
- 铅
- част
- ,char
- анны
- ,ID
- idas
- pole
- /pi
- ónica
- (bt