line breaks inside running text
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 15 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron reacts to line breaks that fall in the middle of a sentence or list item, where the text carries on after the break. It was strongest on those breaks, and it went quiet, even negative, on breaks that come right after a question mark ending a list line. Turning it up made the model fall into long runs of blank lines, but the steered text was otherwise hard to read.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
off Victory|⏎Federal Elections (94-2004): Election Results for the US Senate⏎and the US House of Representatives. Copyright of the Federal⏎Elections
-
early through its center; by Mad and White Rivers; and by Lake Champlain,⏎which affords it many navigable privileges. The soil is good, particularly⏎in those towns below the mountains, and bordering the lake<|endoftext|>“ The L
-
can⏎be fitted together?⏎Using different numbers of sticks, how many different triangles are⏎you able to make? Can you make any rules about the numbers of⏎sticks that make the most triangles?⏎Design an arrangement of display boards in
-
the Kupa Piti Kungka Tjuta, the Senior Aboriginal⏎Women from Coober Pedy, South Australia. Documents their knowledge of atomic⏎tests in the 1950's and 60's and their
-
things to look for⏎helpful and feed back to the team at the end of the task.⏎- Team members cannot show their cards to anyone else on the⏎- A team member can describe what is on their card.⏎- Team members
-
Teleostomi (teleost fish) > Osteichthyes (bony fish) > Class:⏎fish) > Stegocephalia (terrestrial⏎vertebrates) > Tetrapoda⏎(four-legged vertebrates) > Re
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately:ThisIThisIItMaybeItMaybeItThisButITheItMaybeMaybeMaybeMaybeMaybeMaybeIThisButItButMaybeIThisTheThisButIThisIThisMaybeIThisThisMaybeThisThisIButIThisItI
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
strengths and limitations of this theory are.⏎Ready to learn more about Mackinder’s Heartland Theory? Read on…⏎- What is Mackinder’s Heartland Theory?⏎- Why is the Heartland so important?⏎- What are the strengths of the
-
the four most common types of soil and what plants grow best in them, so that you can understand your garden better!⏎Check it out!⏎- What Is Soil?⏎- Mixtures And Compounds⏎- What Is Matter?⏎- Combinations And
-
row)⏎corresponds exactly to the order of data points in the actual data file.⏎
-
and each of the bytes checked on the the ptr array are interpreted as unsigned char for the comparison.⏎- Pointer to the block of memory where the search is performed.⏎- Value to be located. The value is passed
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L14:56 strongly suppresses document ends and list line breaks
- L10:84 strongly suppresses
- L13:10 strongly excites
- L1:1279 suppresses
- L14:1067 suppresses
- L6:67 excites
- L5:331 suppresses
- L9:25 excites
- L9:605 excites
- L8:32 excites
- L3:98 excites
- L2:204 excites
- L10:259 excites
- L0:37 excites
- L14:51 excites colon that introduces a statement
- L6:216 excites
- L5:238 excites
- L14:348 suppresses
- L5:285 suppresses
- L8:88 excites line breaks inside wrapped sentences
- L9:2729 suppresses
- L4:105 excites
- L2:256 weakly suppresses
- L14:415 weakly suppresses
Feeds later layers
- L16:313 strongly suppresses
- L17:829 strongly suppresses
- L18:429 strongly suppresses
- L17:205 strongly suppresses
- L16:190 strongly suppresses end-of-document boundary token
- L16:359 strongly suppresses
- L18:5 strongly suppresses
- L16:1579 strongly suppresses
- L20:346 suppresses
- L19:187 suppresses
- L17:1073 suppresses
- L17:432 suppresses
- L21:130 suppresses
- L16:2348 suppresses line break after a section heading
- L17:921 suppresses
- L18:705 suppresses
- L16:1421 suppresses
- L18:572 excites
- L18:528 suppresses
- L18:886 suppresses
- L17:855 suppresses
- L16:442 suppresses
- L18:3042 suppresses line breaks in wrapped text
- L22:506 suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- ⏎→⏎
- ⏎→→⏎
- :\/\/
- ⏎→→→→⏎
- ⏎→ ⏎
- ⏎ ⏎
- onyms
- _ ⏎
- ) ⏎
- __ ⏎
Suppresses
- 1
- (a
- beck
- (d
- B
- 传闻
- 以下
- 6
- (
- 4