preposition after a noun phrase
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 14 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron lights up on the small connecting word (usually 'of' or 'in') that comes right after a noun and introduces a name or detail, such as 'Battle of St Albans' or 'cells in the central nervous system'. When it was turned up, the model's replies did not show a clear theme.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
official language in the content classrooms, typically mathematics and science. According to Cummins, bilingual education is defined as “the used of two (or more) language of instruction at some point in a student’s school career” (cited in Cre
-
infect the nervous system. It reactivates later in life in people who have suppressed immune systems. The virus damages oligodendrocytes, the cells in the central nervous system (CNS) which produce myelin, the material that
-
⏎Reproduced from https://style.mla.org/whats-new/⏎Author. When a source has three or more authors, only the first one shown in the source is normally given
-
properly. This is the motivation that put the Basel Convention into play.⏎In Cuernavaca, a plant was built and paid for by the US Filter Corporation of Palm Desert, California. The filter was the first granted to an American company making
-
and a minor gentleman, Sir Richard Woodville. Elizabeth’s first husband, was Sir John Grey, and they had two sons before he died at the Second Battle of St Albans. In September 14
-
was 72.⏎A sister, Casey Camp-Horinek, said her brother had cancer.⏎Mr. Camp, a member of the Ponca Tribe of Oklahoma, was a longtime member of the American Indian Movement, organizing more than
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: of, say, in any of the, I have been, perhaps, encountered in some of, I have been, perhaps I, or, now, in, or, perhaps, I, or, now, I, or, I
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
and educational opportunities. The women’s declaration appropriated the language and form of the Declaration of Independence. Considered shocking in its time, many of the aims and goals contained in the Declaration of Sentiments apply to
-
lers’ farm animals destroyed the shoots of new trees and trampled the land. The compacted soil was less fertile and less absorbent. The environmental pressure exerted by the settlers was not only a result of numbers, but also of their subs
-
was required and it was introduced as pitta. Vata, pitta and kapha came to be called the Tridosha.⏎Meanwhile, the theory witnessed expansion in another interesting line. Once the idea of the two-fold causes of diseases
-
the end they don't look at the writer by what he writes, but rather evaluate him by where he lives, or by the location of "the room" from which he writes, as the Peruvian writer Vargas Llosa noted in
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L10:1124 suppresses
- L7:1451 excites
- L12:1823 suppresses
- L6:958 suppresses
- L9:865 suppresses
- L11:775 excites
- L8:1703 excites
- L13:2988 excites
- L11:1381 excites
- L10:155 suppresses
- L7:2503 suppresses
- L12:1531 suppresses
- L5:1428 suppresses
- L10:669 excites
- L13:44 excites inside web addresses and citations
- L13:423 excites
- L8:982 suppresses
- L11:223 suppresses
- L13:1108 suppresses
- L9:2312 weakly excites 'to' after a noun (a role to play)
- L12:958 weakly excites
- L12:1222 weakly suppresses
- L7:685 weakly excites
- L9:392 weakly suppresses "and" inside organisation names
Feeds later layers
- L16:2746 suppresses
- L19:2621 suppresses
- L15:632 suppresses
- L15:1464 suppresses
- L20:55 excites
- L15:2728 suppresses
- L16:1739 suppresses
- L18:68 suppresses
- L17:1167 suppresses
- L15:1851 suppresses 'that' after a noun, 'being' in passives
- L18:1672 suppresses
- L15:817 excites
- L16:1441 excites
- L15:942 excites
- L15:2893 suppresses
- L17:2067 suppresses
- L16:1322 suppresses
- L15:1247 suppresses
- L15:1328 suppresses
- L22:132 suppresses
- L17:2267 suppresses
- L17:812 excites
- L15:396 excites
- L20:1481 suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- 能否
- 所以他
- 能不能
- ledge
- xn
- 甚至
- fk
- ему
- 矩
- 粉色
Suppresses
- @",
- ญ
- onnen
- ์
- above
- ashtra
- :",⏎
- downloaded
- carved
- specified