"in" after superlatives and "one of the few"
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 20 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron wakes up on the word "in" when it follows a phrase like "the best prophet", "the foremost intellectuals" or "one of the few maps". When it was turned up, the model got stuck repeating "and, and, and". Turned down, it wrote ordinary text.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
billion tons in the U.S. alone) over the past 30 years through the use of catalytic-converter-equipped vehicles. Virtually every automotive company in the world today relies on a system that is based on a cellular ceramic substrate technology
-
These characteristics became his signature during his career as one of the foremost intellectuals in the Afro-Asian anti-colonial movement.⏎Education and Political Activism⏎
-
|[Home] [The History of Shirayuki and Sake] [Our Products] Japanese Page|⏎Japan's finest sake in historical perspective⏎Rice was found its way into Japan about 200
-
law. The Holy Prophet being the final law-giving prophet who has come for not just one religion, but for all mankind, would therefore be the best prophet in the sight of Allah. This is what we testify to.<|endoftext|>Those aged over
-
has been home to many people for centuries and people from all part of India have stayed here and have enriched the city with their ethos and customs. Very few cities in India offer the real spirt of India at one place. Delhi is home to
-
scientists have made a surprising discovery: a gigantic aquifer of relatively fresh water trapped in porous sediments lying below the salty ocean.⏎A new book, the second in a series of primers with the Earth Institute imprint, provides an interdisciplinary overview drought
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: if I were to write a book, I would need to have a **displacement difference between the "self" [suggestion, *s} [a] *s [a] *s [a] **s [a]
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
of the class in which he or she is enrolled.⏎The HCPS Office of Exceptional Education ensures that individuals employed as homebound teachers hold a Virginia teaching certificate or be determined eligible for one
-
nation.⏎According to the Title I Director’s Conference (February 2003), “highly qualified” means that all teachers must have full State certification or have passed teacher licensing exam, and hold a license to teach, and that certification
-
Causes of miscarriage can include problems with the mother's uterus or placenta, or abnormalities in the baby's development. In most cases, the
-
congenital illness that results in the degradation of the retina. People with LCA slowly lose all vision, and typically are completely blind by their 20s or 30s. The modified virus used in the therapy carries a replacement gene
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L18:93 suppresses
- L13:1046 suppresses 'in' or 'on' ending a phrase
- L14:348 suppresses
- L15:727 weakly excites
- L12:51 weakly excites
- L14:1091 weakly excites
- L16:245 weakly suppresses
- L17:385 weakly suppresses
- L15:2502 weakly excites the gap in number ranges
- L1:1062 weakly suppresses
- L5:5 weakly suppresses digit after a label like paper or type
- L11:104 weakly excites
- L1:388 weakly excites
- L17:317 weakly excites 'of' after a noun phrase
- L13:3043 weakly excites
- L19:1944 weakly excites
- L14:2957 weakly suppresses
- L9:29 weakly excites opening parenthesis before an acronym
- L10:125 weakly excites
- L15:1738 weakly suppresses
- L18:565 weakly suppresses
- L15:286 weakly excites
- L17:62 weakly suppresses
- L18:598 weakly suppresses
Feeds later layers
- L25:6 suppresses
- L24:205 excites
- L21:393 excites unclear: relative 'that' and list items
- L22:632 excites
- L26:347 excites the 're' in self-reliant and similar
- L27:289 excites
- L27:291 excites
- L26:74 suppresses
- L23:1798 excites
- L21:76 suppresses
- L27:358 weakly excites
- L25:969 weakly excites
- L26:2056 weakly suppresses
- L23:379 weakly excites
- L27:1364 weakly excites
- L27:2924 weakly suppresses
- L26:65 weakly suppresses
- L22:158 weakly suppresses
- L25:2388 weakly suppresses
- L26:307 weakly excites
- L21:2638 weakly excites
- L21:530 weakly suppresses
- L21:395 weakly excites
- L26:836 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- and
- 和
- 性和
- 和
- 、
- or
- và
- 和个人
- hoot
- and
Suppresses
- ernaut
- 一一
- 新规
- 们都
- liers
- со
- irling
- sa
- 也不例外
- chu