"the" after time-period prepositions
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 23 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on "the" right after phrases like "at the beginning of", "in the 1960s" or "since the middle of", where a decade or century is about to be named. Turning it down gave generic text about "the number of the day", and turning it up gave strings of digits.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
known) refers to more than just learning via computer. Edtech has existed in various forms for as long as education itself has been around. At the beginning of the 20th century, the pencil
-
aint-Laurent, known as Yves Saint Laurent was a French fashion designer, one of the greatest names in French fashion in the 20th century. In 1985, Caroline Renn
-
was elected to the National Academy of Sciences and awarded the National Medal of Science, she rallied for<|endoftext|>Inspiration from Angolan traditional designs⏎Since the middle of the 1980's I have been analysing mathematical properties of traditional
-
psychedelic drugs, such as LSD and magic mushrooms, cause harm to the brain or other organs. Ever since it was suggested that their use can trigger psychosis back in the ‘60s, ideas that they
-
for the pilgrimage to Mecca, up to its modernisation by the construction of the railway at the start of the 20th century.⏎Criterion (iii): The site of Al-Hij
-
Follow Us on Google+ Watch videos on our Youtube channel⏎While hip and knee joint replacement surgery or arthroplasty has grown increasingly common since its advent in the early 1960s with a high degree of patient satisfaction, spine
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the #2.5 #3.3 #3.3 #3.3 #3.3 #3.3 #2.5 #3.3 #3.3 #3.3 #3.3 #3.
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
F. Kennedy, the Beatles seemed to be the perfect tonic to America's melancholia.⏎The United States in early 1964 was a society on the brink of profound transformation.⏎Rock-and-roll music, through the auspices of Elvis
-
told ITV News Central they are bracing themselves for a potential outbreak of measles. Officials in Herefordshire fear the epidemic in Wales means an outbreak is 'on the doorstep'.⏎They say that so far this year they've already had the same number
-
show that the flu costs businesses approximately $7 billion per year due to the nearly 111 million work days lost. Norovirus cases are also on the rise this season, and the CDC reports that foodborne norovirus illness accounts
-
warmer, which is why May is the perfect month for Motorcycle Safety Awareness,” said U.S. Transportation Secretary Anthony Foxx. “Fatal crashes with motorcycles are on the rise. We all need to be more aware of motorcyclists in order to
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L20:673 suppresses
- L19:577 excites
- L21:328 suppresses
- L17:30 suppresses
- L19:139 suppresses
- L20:85 excites
- L19:184 suppresses
- L15:574 weakly excites
- L18:580 weakly suppresses
- L22:284 weakly suppresses
- L20:361 weakly excites
- L2:1657 weakly excites
- L21:339 weakly suppresses spot just before a year
- L16:704 weakly excites
- L18:454 weakly excites
- L10:58 weakly excites
- L21:614 weakly excites
- L16:2731 weakly excites
- L20:508 weakly suppresses
- L4:912 weakly suppresses
- L16:792 weakly excites "the" inside a prepositional phrase
- L12:2611 weakly excites
- L22:2636 weakly excites
- L16:419 weakly excites
Feeds later layers
- L26:611 strongly excites
- L27:306 strongly excites
- L27:113 suppresses
- L26:2953 suppresses
- L24:724 excites
- L26:329 suppresses
- L25:253 excites
- L27:448 suppresses
- L27:287 excites
- L27:2172 suppresses
- L25:146 suppresses
- L26:103 suppresses blank space before a number
- L25:2915 suppresses
- L26:172 suppresses
- L25:95 weakly excites
- L27:1117 weakly suppresses
- L26:57 weakly suppresses
- L27:952 weakly suppresses
- L24:381 weakly excites
- L24:1238 weakly suppresses blank space before a number
- L27:992 weakly suppresses
- L24:697 weakly excites
- L25:921 weakly suppresses
- L25:329 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- reau
- arden
- íc
- ⏎ ⏎
- ⏎
- ittle
- ney
- els
Suppresses
- #⏎
- #
- #⏎⏎
- //(
- #
- //{⏎
- cent
- :\/\/
- coinc
- //⏎