end of document boundary token
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 21 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the invisible marker the training text uses to show that one document has ended and another begins. When turned up, the model tended to start writing code-style backtick quotes, which is a weak match for what the neuron seems to predict.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
er is 800 mum x 150 mum, and the delay time from the clock to Sum31 measured 0.9 ns.<|endoftext|>Hard work and smart work are two distinct approaches to achieving a goal. Hard work
-
the GUI which displays and analyze the possibility of fire happening. The system can display the fire location and provides early warning to allow occupants to escape the building safely.<|endoftext|>Ross and Wilson Anatomy and Physiology in Health and Illness : Anne Waugh :
-
shows promise for the future of electric vehicles and the aerospace industry. This research is supported by the Grainger Center for Electric Machinery and Electromechanics.<|endoftext|>This story now⏎IN History & Culture ON 20 Oct, 2
-
that already have been demonstrated experimentally. Thus, the results presented here suggest that such improvements might be realized in the near term for important, special-purpose applications.<|endoftext|>Communitarianism as a group of related but distinct philosphies began in
-
25% for conditions 1, 2 and 3, respectively. Thus, monocular depth cues aid the detection of moving objects by moving observers.<|endoftext|>Cut flowers need proper care to prolong the length of time you can enjoy them.
-
mann machines (RBM), deep belief networks (DBN) and deep convolutional neural networks (CNN). In recent years, CNN-based methods have gained popularity<|endoftext|>typedef unsigned int size_t;⏎This type is used to hold size definitions. Since
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: what is the result of the sum of the number of all possible permutations of the numbers 1,2,3,4,5,6,7,8,9 on the number 8. In the answer, I just have
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
.codeguru.com/algorithms/factorial.shtml I don't happen to think that the code there is easily understandable though...⏎January 10th, 2004 01:3
-
1|⏎|-||Announced||May 4, 1990|⏎|-||Restored||August 21, 1991|⏎|-||Total||64,589 km2 (124
-
as 14,000 years. It came as a shock to the couple, who had recently moved from Washington state to their new home in Vegas.⏎A couple of days later, police and investigation officers showed up on their door,
-
See Speed Force Powers section.) However, the Speed Force is not the source from which other DC characters with super speed such as Superman or Captain Marvel derive their powers.⏎Writer John Byrne maintained modest abilities for the speedster character Danny Hilltop in his
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L20:1889 excites
- L13:1325 excites
- L17:424 weakly suppresses sentence-ending full stops
- L6:16 weakly excites
- L5:2174 weakly excites
- L10:725 weakly suppresses
- L3:493 weakly excites
- L7:2083 weakly suppresses
- L6:13 weakly excites
- L19:2031 weakly excites line breaks in web footers and metadata
- L2:768 weakly suppresses
- L18:5 weakly suppresses
- L4:327 weakly excites
- L19:997 weakly suppresses
- L3:1175 weakly excites
- L20:8 weakly suppresses
- L10:923 weakly suppresses
- L20:44 weakly excites
- L9:1376 weakly excites
- L19:2057 weakly suppresses
- L19:501 weakly suppresses
- L9:1489 weakly suppresses
- L19:2250 weakly suppresses
- L10:952 weakly excites
Feeds later layers
- L26:1853 excites
- L25:711 excites
- L26:2738 excites
- L25:878 excites
- L23:577 weakly excites
- L27:2391 weakly excites
- L25:255 weakly excites pieces of split scientific or place names
- L25:1880 weakly suppresses
- L26:47 weakly excites
- L25:2541 weakly excites 'set' as in 'set up'
- L25:1914 weakly excites
- L26:2284 weakly excites
- L27:2964 weakly excites
- L25:2703 weakly suppresses
- L23:3020 weakly suppresses
- L23:3025 weakly excites
- L26:1082 weakly excites
- L23:793 weakly suppresses
- L26:1188 weakly excites
- L23:374 weakly suppresses
- L25:484 weakly suppresses
- L25:443 weakly excites
- L24:369 weakly excites
- L25:2207 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- ``
- ``
- ```
- ```
- ``,
- ```⏎
- ``(
- \`
- ``⏎
- 〕
Suppresses
- ",⏎
- ',⏎
- ',⏎
- '}}⏎
- ', ⏎
- ',⏎⏎
- ", ⏎
- ',{⏎
- &⏎
- пом