the digit 3 in dates
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 24 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the number 3 when it starts a day of the month, like 'May 3' or '30 July'. Turning it up did not make the writing talk about dates, so the steering results don't confirm the pattern.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
Jake Matijevic."⏎- Curiosity on Mars sits on rocks similar to those found in marshes in MexicoTue, 30 Oct 2012, 11:07:1
-
(19 April - 22 December 1947)⏎- The IG Farben Trial (27 August 1947 - 30 July 1948)⏎- The Hostages Trial (8 July
-
status silver objects.⏎Elements of the hoard will be on display for the first time at King’s Museum, University of Aberdeen from 20 January to 31 May 2015
-
hemisphere and 2011 southern hemisphere influenza seasons.⏎During weeks 29 to 30 (17 July 2011 to 30 July 2011), National Influenza Centres (
-
947)⏎- The Judges' Trial (5 March - 4 December 1947)⏎- The Pohl Trial (8 April - 3 November 1947)⏎- The Flick
-
65)⏎Historical Note: The 84th Indiana Regiment was formed in Richmond, Indiana (Wayne Co.), and mustered into service on 3 September 1862. It left for Covington, Kentucky, where
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the number 222222222222222222222222222222222222222222222
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
organisms based on studies of their different characteristics.⏎Two word system developed by Carolous Linnaeus to name species, first word identifies the genus of the organism, the second word is often a descriptive word that describes a characteristic of the organism.⏎Next
-
texts, where it is attested already in the Kojiki 古事記 (712) 8, in the Nihon Shoki 日本書紀 (720) 9 and in the Hizen-no-k
-
learning!⏎- Line up the bowls on an elevated surface (coffee table, ottoman, TV stand). Put one ball in the first bowl, two balls in the second, and so on. Count aloud as you go.⏎- Do a color
-
authoritarian Imperial Germany from dominating the Contindent in World War I. Next America pervented totalitarian and genocidal NAZI Germany from dominating the Continent in World war II. And then after the
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L23:599 excites
- L19:2013 excites
- L20:1928 excites
- L23:416 suppresses
- L19:1985 excites
- L19:1260 excites
- L18:2251 suppresses
- L21:1239 weakly suppresses
- L21:3057 weakly suppresses
- L6:7 weakly suppresses
- L20:438 weakly excites
- L13:1005 weakly excites
- L16:2896 weakly excites
- L16:2941 weakly suppresses
- L1:2134 weakly excites
- L20:2449 weakly suppresses
- L13:2087 weakly excites
- L19:3028 weakly suppresses
- L6:8 weakly suppresses
- L19:1958 weakly excites
- L1:779 weakly excites
- L23:2629 weakly excites
- L10:3 weakly excites
- L13:2484 weakly excites
Feeds later layers
- L26:834 suppresses
- L27:189 excites
- L27:992 suppresses
- L27:352 suppresses
- L25:379 suppresses
- L26:598 excites spot just before a year or century
- L27:132 excites
- L27:180 suppresses
- L27:328 excites
- L26:848 excites
- L25:1452 excites
- L26:233 suppresses
- L26:270 excites
- L25:2298 weakly suppresses
- L27:902 weakly excites
- L26:682 weakly suppresses
- L27:555 weakly suppresses
- L27:1522 weakly excites
- L27:652 weakly excites
- L26:1083 weakly suppresses
- L25:1749 weakly suppresses
- L26:2240 weakly suppresses
- L26:2515 weakly excites
- L27:1393 weakly suppresses predicting the word two
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- 1
- 싱
- bat
- �
- ency
- \Migration
- rin
- trap
- hear
- rhs
Suppresses
- cont
- pta
- posta
- ʒ
- 2
- 2
- Olymp
- -two
- ап
- paylaş