the word 'that' introducing a clause
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 10 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron reacts to the word 'that' when it links a phrase to a following clause, as in 'so that' or 'those that'. It goes strongly negative on 'that' used as a pointing word after a verb, like 'tell that story' or 'check out that lesson'. Turning it up did not produce a clear theme in the model's writing.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
Sustainability 2013, 5(7), 3006-3008; doi:10.3390/su5073006⏎I am honored to introduce this
-
possess, so that you, your children and their children after them may fear the Lord your God as long as you live by keeping all his decrees and commands that I give you, ad so that you may enjoy log life” (Deuter
-
Team⏎Our team consists of highly trained and skilled professionals that are in these sorts of children problems. We will give you the best possible advice there is, so that you could help your child as fast as you can. We are here for you
-
may be nothing about (var1 x)⏎> that changed, and there may be nothing about var11 . var1 $ x
-
We hold these Truths to be self-evident, that all Men are created equal, that they are endowed by their Creator with certain un
-
and its hardiness allows the plant to out-grow and out-compete native plants. This, in turn, affects the organisms—from insects to birds and mammals – that have evolved in tandem with Utah’s plants. Myrtle spurge is also incredibly
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the first one, the first one, the first one, the first one, the first one, the first one, the first one, the first one, the first one, 1, 1, 1, 1,
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
in this film from start to finish. It was the first of its kind to have the depth of character, careful attention to story, original music that helped tell that story and superb artistry.”⏎Premiered at Hollywood’s Carthay Circle
-
video in their curriculum. Reading materials and lesson ideas are available. I recently used a lesson that my colleague created directly from the provided materials. You can check out that lesson here. The lesson included some background
-
smoking program "really works" and has "a 21% quit rate." These statements are contrary to findings from two recent independent studies. But try telling that to U.S. health officials.⏎According to the Centers for Disease Control (CDC
-
in policy that could make the Moon and its surrounding space a more important part of the equation.⏎And our source tells us that part of that plan is deploying RESOLVE on the lunar surface in 201
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L4:468 excites "that" after "than" or "like"
- L8:14 excites
- L9:1752 suppresses
- L6:1080 excites "that" as a pointing word
- L4:671 excites 'that' starting a new clause
- L4:270 weakly suppresses
- L6:925 weakly excites
- L1:2290 weakly suppresses
- L9:1155 weakly suppresses
- L4:1762 weakly suppresses "this" in time and context phrases
- L9:284 weakly excites
- L2:181 weakly suppresses
- L7:1253 weakly suppresses
- L8:3050 weakly suppresses
- L0:237 weakly suppresses
- L9:1088 weakly suppresses
- L3:1196 weakly excites
- L0:918 weakly suppresses
- L7:1252 weakly suppresses
- L4:365 weakly excites
- L1:2334 weakly suppresses
- L5:2680 weakly excites
- L0:1086 weakly suppresses
- L7:181 weakly excites
Feeds later layers
- L12:521 suppresses
- L17:221 suppresses
- L15:470 suppresses the word 'this' after a preposition
- L11:199 suppresses
- L17:2011 suppresses
- L13:1007 suppresses
- L14:1807 suppresses
- L19:2547 suppresses
- L16:2861 suppresses
- L15:1957 suppresses
- L17:1821 suppresses
- L12:2701 suppresses
- L17:1816 suppresses
- L15:1323 suppresses
- L16:2466 suppresses
- L15:1187 suppresses
- L14:2263 suppresses the word 'same' as a referring word
- L14:2444 weakly suppresses
- L18:1228 weakly suppresses
- L14:2220 weakly suppresses "this" pointing back to a thing
- L19:494 weakly suppresses
- L12:849 weakly suppresses
- L16:815 weakly suppresses 'the' before a noun mid-sentence
- L20:1175 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- cứng
- ed
- care
- Care
- "/"⏎
- intervene
- ue
- unders
- 记者采访
- conform
Suppresses
- !"; ⏎
- ATTLE
- TINGS
- .";
- ."; ⏎
- ...";⏎
- hg
- Aws
- WHERE
- monds