'is' right after 'there'
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 9 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the word 'is' when it comes straight after 'There', as in 'There is a high demand'. Turning it up made the model's text wander into vague, general statements, but the link to 'there is' is only visible in the examples.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
the fifth most commonly spoken native language in the world.⏎- Arabic is the liturgical language of Islam.⏎- There is a high demand and low supply of Arabic-speakers in the Western World.⏎-
-
to enhance their appearance.⏎Only a few gemstones that are called organic come from plants and animals. These gemstones are amber, ivory, and pearl. There is a category of gemstones known as plastic. They do not come from nature and
-
ethnocentrism, consumer ethnocentrism is specific to consumers of a particular country. Ethnocentrism refers to general groups of people, wherein there is an "in-group" and an "out-group." In terms of consumer eth
-
Apples are beneficial in Diabetes⏎Apple are very effective in the treatment of diabetes. Apples contain a rich, natural therapeutic substance called pectin. There is p
-
are displayed here.<|endoftext|>Epigenetics is the study of changes in the regulation of gene activity and expression that are not dependent on alterations in gene sequence. There is a rapidly growing realization that complex diseases ranging from asthma, diabetes, addiction, schizophrenia
-
, such as through CT scans, MRI, X-rays, ultrasound, or nuclear tests like the Positron Emission Tomography (PET) Scan. There is also imaging that is used to see parts of the body that are undetectable
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: If you are, for example, a member of the U.S. government, then you are free to... (but not restricted) by the U.S. government. What does this mean? Now, I have been thinking about it,
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
Aetna) is an active stratovolcano on the east coast of Sicily, close to Messina and Catania. It lies above the convergent plate margin between the African Plate and the Eurasian Plate. It is the tallest
-
Contentment⏎Shakespeare’s play Richard III, Act-I, Scene-I, Lines 1-4, begins with the following famous stanza:⏎“Now is the winter of our discontent⏎Made glorious summer by this son of York;⏎And all
-
and where to look.⏎The Space Station orbits Earth about 15 times per day. With your unaided eyes, it looks like a bright star moving across the sky. But it doesn’t blink like an airplane.⏎To find out when
-
few weeks longer for some cats, but most kittens will have their baby teeth out by the 8th week.⏎Check; does your kitten have all of his temporary teeth out yet? He is probably around 8 to 10 weeks of
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L5:2153 suppresses
- L5:1094 suppresses
- L4:341 weakly suppresses
- L7:1760 weakly suppresses
- L4:1579 weakly excites
- L8:1632 weakly excites
- L6:1849 weakly excites
- L8:1023 weakly excites
- L4:636 weakly suppresses
- L3:2984 weakly excites unclear: mid-word subword pieces
- L8:1300 weakly excites
- L7:2921 weakly excites
- L8:2008 weakly suppresses
- L6:5 weakly excites
- L0:689 weakly suppresses
- L7:1691 weakly suppresses
- L1:2622 weakly suppresses
- L3:544 weakly suppresses opening parenthesis in lists and asides
- L1:2600 weakly suppresses
- L5:766 weakly suppresses
- L3:216 weakly suppresses empty slot before a figure number
- L3:1382 weakly suppresses
- L8:827 weakly excites
- L4:1418 weakly excites
Feeds later layers
- L16:2913 excites
- L20:1404 excites
- L17:938 excites
- L11:1528 suppresses
- L11:773 excites
- L16:2448 excites
- L11:979 excites
- L14:534 suppresses
- L25:1994 weakly excites
- L21:2458 weakly excites
- L19:1983 weakly excites
- L20:2682 weakly excites
- L13:1732 weakly excites
- L21:1222 weakly excites
- L21:732 weakly excites
- L11:1962 weakly excites
- L17:2820 weakly excites
- L16:1099 weakly suppresses
- L20:2326 weakly excites
- L14:1845 weakly suppresses
- L10:728 weakly suppresses
- L12:534 weakly suppresses
- L26:2817 weakly excites
- L24:2131 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- OTS
- .createObject
- "There
- angs
- SACTION
- Dion
- DH
- נן
- inction
- There
Suppresses
- 来源于
- inee
- 伙伴
- favor
- oeff
- 坐在
- 过往
- 宝
- ipl
- 得益于