comma after a cause or effect clause
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 25 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on a comma that comes after a sentence describing a cause, such as "too much sodium causes your body to retain water," where the sentence is about to add what it leads to. Turning it up made the model repeat the word "making", which fits that it expects a phrase like "making" or "leading" to follow.
Written by Claude from this neuron's activations and steering outputs. Confidence: high. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
that is the cause of the health concerns.* Sodium is essential in the body to help regulate fluids and blood pressure. Too much sodium causes your body to retain water, which puts extra strain on your heart and blood vessels. By lowering the amount of
-
. Eventually, however, it may fail to keep up, resulting in type 2 diabetes.⏎Your genes, excess body fat and lack of exercise cause insulin resistance, but eating the wrong kinds of carbohydrates seems to worsen the problem. In one
-
. By performing weight-training exercises, you can build strength in your muscles, increase the amount of lean muscle in your body, manage your weight more easily and
-
. However, if you’re able to use<|endoftext|>Need an excuse to treat yourself to dessert? A daily dose of chocolate or cocoa powder could help lower blood pressure, say the authors of a new systematic review by the Cochrane Library.⏎
-
agricultural practices wherein the soil is consistently turned over. By doing so, each layer of the soil gets exposed to the sun, and leads to soil dehydration. Soil comprises various bacteria, fungi and worms that are
-
are never filled. Such shortcomings in medication adherence can decrease the effectiveness of a treatment. This increases the likelihood that a condition will worsen, even to the point of death.⏎But these figures fail to consider the numerous challenges
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: I always associate the numbers 7 and 8 with the numbers 8 and 9 always again, these numbers always hold these same numbers even if these numbers are also written in a different format even if these numbers also can be written in
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
, but in one’s chest, one’s hands, and one’s feet, all together.”⏎Long describes Homer’s overall style to be that of ‘sensuous immediacy’, writing: ‘Homer delights in the vibrancy of every
-
am is introduced to Frances Henri, a lady teacher who instructs the girls in sewing, as his new student. Sensible, industrious and desolate, this girl slowly gains his admiration and Mad
-
He became a billionaire by founding the Standard Oil Company, which was the largest oil company in the world at the time. Rockefeller was also a great philanthropist, donating millions of
-
in Washington, D.C., and became a house painter, enjoying a reasonably comfortable life. Turnage had a more hardscrabble existence in New York City, where he worked as a watchman, waiter, and other occupations.⏎Their narratives
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L19:1036 suppresses
- L18:366 excites comma inside numbers and dates
- L22:1425 suppresses
- L14:35 suppresses
- L22:144 suppresses
- L17:118 excites comma after a clause ending
- L20:476 excites commas that lead into a continuation
- L18:892 suppresses
- L22:1414 suppresses
- L16:846 excites comma opening an aside or explanation
- L19:726 suppresses comma inside large numbers
- L19:944 suppresses
- L16:240 excites comma after 'sometimes' or 'the truth'
- L18:1688 excites
- L22:940 suppresses
- L21:200 suppresses
- L19:291 weakly suppresses 'and' joining two full clauses
- L19:1380 weakly excites
- L19:1361 weakly suppresses
- L19:2175 weakly suppresses
- L17:3021 weakly suppresses
- L19:94 weakly excites
- L19:383 weakly suppresses commas after a clause or list item
- L20:1832 weakly suppresses 'as' introducing a reason
Feeds later layers
- L27:410 excites
- L27:677 excites
- L26:686 excites
- L26:252 excites
- L27:1890 excites
- L26:1304 weakly excites
- L27:1545 weakly excites
- L26:708 weakly excites
- L26:1296 weakly suppresses word-start 'mult' fragment
- L26:979 weakly suppresses
- L26:2677 weakly excites
- L27:1148 weakly excites
- L27:566 weakly excites
- L27:409 weakly suppresses
- L26:432 weakly excites
- L26:2383 weakly suppresses
- L27:2982 weakly excites
- L26:2881 weakly suppresses
- L27:248 weakly suppresses
- L27:1112 weakly excites
- L26:2104 weakly suppresses
- L26:764 weakly suppresses
- L26:1444 weakly suppresses
- L27:1515 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- making
- making
- giving
- suggesting
- leading
- allowing
- implying
- leaving
- indicating
- confirming
Suppresses
- 体
- hari
- APTER
- 世
- ownt
- stands
- 就好
- 쯤
- ote
- estroy