number right after a year or range word
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 22 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the blank spot right after a word like 'from', 'to', or 'between' where a number, usually a year, should come next. When it was turned down, the model began writing strings of zeros and ones.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
the health and living habits of 15,000 residents in six Brazilian capitals over a 10-year period between 2008 and 2018, researchers from the Longitudinal Study of Adult Health (
-
ics that gave the suspect shots<|endoftext|>Classic DACB CollectionAll articles created or submitted in the first twenty years of the project, from 1995 to 2015.⏎Jacob Osasona was born in the 18
-
oted"? Overwhelmingly for Obama. The "popular vote" (that is, the share of the overall votes cast) is running 85% to 15% in favour of the Democratic candidate. At the time of writing,
-
S., & Ekman, S.L. (2002). Caregiver singing and background music in dementia care. Western Journal of Nursing Research, 24(2), 195-216. My approach
-
be explored in depth in an Education Program Session today at 7:30 a.m. and again at 2:00 p.m. in rooms 393-396 of the Ernest N. Morial Convention Center
-
105 of whom had been exposed to gestational diabetes (mean age, 5.9 years; 50.5% boys) and 38 who were unexposed (mean age, 6.8 years
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: 00-10-00-10-00-10 00-10-00-10-00-10 00-10-00-10
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
at his code, x would never be zero, that wasn't the issue. The fact is that it is pointless even having the x if you set total to 0 at the start because total will never increase, and that's even if you
-
elements of the array in reverse, we need to use a loop that will iterate from length - 1 to 0. We can use a for loop or a while loop to write this
-
to check who it was that posted it, and I can't see just now anyway :P)⏎void genFactorial(int x, int t)⏎int total = t; // you probably wouldn't even need total now⏎void main(void)
-
that can run all the commands with one message from you.⏎% filename simple.m⏎ini=1.; %set the variable ini to 1.⏎y0=ini; %set the initial condition to ini
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L20:567 excites
- L20:205 excites
- L19:3047 suppresses
- L20:1479 excites
- L18:115 suppresses
- L14:827 suppresses open bracket after a value or citation
- L19:248 excites
- L13:287 suppresses
- L21:1595 suppresses
- L21:3007 weakly suppresses
- L13:818 weakly suppresses
- L18:1259 weakly suppresses
- L18:2519 weakly suppresses
- L18:930 weakly excites
- L20:947 weakly excites
- L3:385 weakly suppresses
- L10:272 weakly suppresses
- L20:1493 weakly excites
- L14:1272 weakly excites
- L21:253 weakly suppresses
- L8:2755 weakly excites
- L20:780 weakly excites 'of' before a measured value
- L16:1142 weakly excites
- L14:442 weakly excites
Feeds later layers
- L25:46 excites
- L23:198 suppresses
- L27:396 suppresses
- L26:329 excites
- L26:611 suppresses
- L27:287 suppresses
- L27:113 excites
- L25:473 suppresses zeros inside large round numbers
- L23:250 suppresses
- L27:153 excites the spot just before a historical number
- L26:413 suppresses
- L27:691 suppresses
- L26:2953 excites
- L27:48 excites
- L25:95 excites
- L25:253 suppresses
- L27:2172 excites
- L27:163 excites
- L26:137 excites
- L27:238 suppresses
- L25:854 weakly suppresses
- L26:216 weakly suppresses blank slot before a year or number
- L23:478 weakly excites
- L26:667 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- ovation
- angkan
- -awesome
- lients
- .Flag
- 赘
- изации
- agus
- arium
- rage
Suppresses
- 0
- -
- 1
- false
- "-
- ;-
- false
- 0
- true