'at the beginning/start' time phrases
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 13 of Qwen3-0.6B. Its strongest snippets are mixed: they peak on different tokens. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on words that mark a point in time, especially phrases like 'at the beginning' or 'at the start' of something, and 'at the time'. When turned down, the model's text got stuck in loops about a story or a girl on a bus, and turning it up pushed the model into vague, rambling sentences.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
and potentially create water savings for the school district.⏎- Many math lessons can be taught when school water use is audited. Conducting a preliminary audit at the beginning of the school year and a post audit before the end of the school year allows
-
of years after the last recorded volcanic eruption of Mount Agung dozens of miles away.⏎Wayan Lebih, 55, was a young boy at the time and remembers gathering co
-
cided lesson plans and syllabus completion plans⏎Any good integrated learning system allows the school management to set out lesson plans and decide syllabus completion deadlines at the beginning of the academic year. Once these are pre-decided, teachers can then
-
the e-learning market in the United States is expected to grow $6.8 billion by 2015, up from $2.9 billion in 2010. According to Sanjeev Ahuja, the vice president
-
of watersheds and drinking water systems across the Midwest and Southern United States. This May 2010 report summarizes scientific information that has emerged since the publication of our initial report and includes more recent monitoring data.⏎Approximately 75
-
, but also the way people will resist as part of unlearning. Snyder’s-Lance zeroed in on the ways managers were expressing resistance and encouraged it early in the process to help them “unlearn” the response patterns that were prohibiting them
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: I was given a puzzle, I had a collection of a few numbers, I noticed that I had 3, I had a, I was given a, I was given the, I had a, I was told, I had a
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
was there from the start; as a baby, Chopin is said to have been moved to tears by his mother’s singing.⏎Chopin started playing piano at age 5 under the tutelage of his sister, Ludwika, and
-
are indelibly etched in the memory of those viewing his work. Born in New York in 1882, Hopper showed early interest in art, particularly drawing, and went on to study illustration and painting. With
-
Christian emperor of Rome, ordered to build a basilica on Vatican Hill, on the site of the present St. Peter's Cathedral. Though a small shrine already existed on the site, it was replaced by a new church that was completed around
-
were excited to see them selected to support the Army Game Studio on the game."⏎Though development of the game didn't start until last year, Laughlin actually began researching the prospect of
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L12:1930 excites
- L12:1313 excites
- L12:2898 suppresses
- L11:389 suppresses
- L12:650 suppresses
- L12:742 excites
- L12:430 excites
- L12:204 suppresses
- L11:764 suppresses
- L12:1516 suppresses
- L8:2618 suppresses
- L9:563 excites
- L11:1819 suppresses
- L12:2857 suppresses
- L12:179 suppresses
- L10:3015 excites
- L8:1228 excites
- L12:587 suppresses
- L11:1742 suppresses
- L12:518 weakly excites
- L12:1583 weakly suppresses
- L9:727 weakly excites
- L12:193 weakly suppresses
- L7:1104 weakly suppresses
Feeds later layers
- L19:457 suppresses
- L14:184 excites
- L16:885 suppresses
- L14:170 suppresses
- L15:1825 suppresses
- L25:1396 suppresses
- L15:1241 excites
- L18:2514 suppresses
- L21:1691 suppresses "at" in "at first" and "initial" words
- L15:193 excites
- L15:2918 suppresses
- L17:2922 suppresses
- L15:2013 suppresses
- L15:2881 suppresses
- L24:2617 suppresses
- L14:324 suppresses
- L14:2646 suppresses
- L14:2329 excites
- L15:1903 suppresses
- L14:1695 suppresses
- L14:1557 suppresses
- L18:3044 excites
- L15:446 excites
- L21:1453 suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- YL
- indemn
- ffee
- ornado
- 髯
- \""
- cos
- yl
- nữa
- độc
Suppresses
- initially
- 初始
- 最初
- 最初的
- initial
- Initial
- initial
- inicial
- →initial
- .initial