'that' after reporting verbs
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 1 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the word 'that' when it follows a verb of saying or showing, such as 'asserts that', 'notes that' or 'showed that'. Turning it up did not make the model's writing obviously follow this pattern.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
5-16, 1972. Representatives of 113 governments attended and adopted the Stockholm Declaration on the Human Environment, which asserts that humans have a fundamental right to an environment that permits a life of dignity and well
-
in there” to “restore fusion actions to [their] former glory.”⏎Supersymmetry and Q-balls are thus far only theoretical, although Cox asserts that he and the physicists at CERN will prove or disprove their existence by
-
ists contend that no matter what the amount used in dental fillings, the mercury is toxic. At the same time, the American Dental Association (ADA) asserts that only a small percentage of individuals exhibit allergies to mercury, and that once mercury is
-
these measurements are used to help identify the species of bat.⏎However, a paper by an international team of researchers, published in the Journal of Applied Ecology, asserts that poor standardisation of acoustic monitoring limits scientists’ ability to collate data.⏎Kate
-
in comparing the post-tsunami scenario with the pre-tsunami situation, particularly with regard to coastal landforms and characteristics of soil and water.⏎Noting that the studies included Madurai, Coimbatore, Salem and Pondicherry
-
, which diminishes a community’s ability to tackle complex problems, including poverty, as well as its ability to generate high-quality economic development.⏎The study also notes that higher poverty rates translate into higher costs for taxpayer-funded public assistance programs.⏎- Download
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: "The first of the book, "The Art of Computer Programming, by Marko's " -" -" -" -" -" -" -" -" -" -" -" -" -" -" -
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
boline alkaloids (which are monoamine oxidase inhibitors, or MAOIs) with N, N-Dimethyltryptamine (DMT) that produce the hallucinogenic effects of the brew.⏎Some say that the value of
-
2009, strains of K. pneumoniae with gene called New Delhi metallo-beta-lactamase (shortened NDM-1) that even gives resistance against intravenous antibiotic carbapenem, were discovered in India
-
MET protein.⏎MET is a receptor tyrosine kinase (RTK) that is produced as a single-chain precursor. The precursor is proteolytically cle
-
cheadlines⏎Google has started a new page on Google Plus to share their vision of
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L0:1224 suppresses
- L0:178 suppresses
- L0:1301 weakly suppresses
- L0:2204 weakly excites
- L0:502 weakly excites
- L0:36 weakly suppresses
- L0:40 weakly excites 'of' in 'sort of' hedges
- L0:1311 weakly suppresses
- L0:302 weakly suppresses
- L0:453 weakly suppresses
- L0:2528 weakly excites
- L0:1764 weakly suppresses
- L0:1837 weakly excites
- L0:2278 weakly suppresses
- L0:277 weakly suppresses
- L0:2027 weakly suppresses
- L0:2230 weakly suppresses
- L0:729 weakly excites 'st' in dates like 'June 1st'
- L0:1025 weakly excites
- L0:612 weakly excites
- L0:386 weakly suppresses
- L0:510 weakly suppresses
- L0:879 weakly excites
- L0:2475 weakly suppresses
Feeds later layers
- L3:379 excites 'that' after 'the good news is'
- L4:227 excites
- L21:1606 weakly suppresses
- L2:505 weakly excites
- L4:223 weakly excites
- L11:1827 weakly excites
- L6:918 weakly excites
- L15:1020 weakly excites
- L10:2554 weakly suppresses
- L25:1364 weakly excites
- L20:1715 weakly excites
- L10:63 weakly suppresses
- L11:2031 weakly excites
- L18:2104 weakly suppresses
- L17:150 weakly excites
- L4:2757 weakly excites
- L19:1177 weakly excites
- L9:2817 weakly suppresses
- L18:2902 weakly excites
- L7:1833 weakly suppresses
- L15:1792 weakly suppresses
- L11:1723 weakly suppresses
- L20:533 weakly excites
- L15:1345 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- ônica
- ónica
- brace
- ónico
- onica
- awn
- rego
- ereal
- Alonso
- arus
Suppresses
- .SelectCommand
- 的表现
- held
- 颊
- 流逝
- cố
- 的地
- 的学习
- flap
- fet