commas inside lists before 'etc'
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 26 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on a comma that sits in the middle of a list, such as 'curb cuts, ramps, restrooms, etc.' It is at its strongest right where the next item is about to come or the list is about to end. When it was pushed up, the model got stuck repeating words like 'either', and when pushed down it repeated 'etc'.
Written by Claude from this neuron's activations and steering outputs. Confidence: high. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
XML-based data on the server, and to transform it into plain-vanilla HTML, XHTML, or other formats (PDF, plain text, rich text files, and so forth) for delivery to Web clients. This helps sidestep the
-
3. Belonging/Social—family, affection, friends, relationships, etc.⏎4. Esteem—achievement, recognition, independence, responsibility, reputation, respect, etc.⏎5. Self-Actualization—realizing potential, self
-
.⏎b. (of two or more attributes of an object) unable to belong to the object simultaneously; inconsistent.⏎5. (of positions, functions, ranks, etc.) unable to be held simultaneously by one person.⏎6. Med.of or
-
00-342-3736.⏎Ask such questions as whether the service location is physically accessible (curb cuts, ramps, restrooms, etc.) as well as whether there is a Telecommunication Device for the Deaf
-
problems, finding the roots of nonlinear equations, functions written in Octave's own language, or by using dynamically loaded modules written in C, C++, Fortran, or other languages.⏎Donations to support the software can be made at https://
-
conservation of electric charge. There are also many approximate conservation laws, which apply to such quantities as mass, parity, lepton number, baryon number, strangeness, hypercharge, etc. These quantities are conserved in certain classes
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc etc等等等等等等等等等等等等等等等等等等等等等等等
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
wind 雨 : rain 声 : sound⏎Meaning for Line 4 花 : flower 落 : fall 知 : know 多 : how many 少 : missing⏎Background Story A
-
al., 2008), it’s obviously extremely successful, although it should be noted that the virulence of each Wolbachia strain is different (Min & Benzer, 1997), and each species is has a
-
at DARPA.⏎The first connection came 4 years later in October 1969, between UCLA (Los Angeles) & the Stanford Research Institute in Menlo Park, California (.. which would become part of Silicon Valley, just south
-
Halton Chesters, Northumberland)⏎The Romano-British name for the Rudchester fort is recorded in two of the major classical geographical works, the Notitia Dignitatum of the late-fourth/early-fifth centuries and
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L24:1087 strongly suppresses
- L21:2512 strongly excites
- L25:162 strongly excites
- L18:2938 strongly excites
- L19:1265 strongly suppresses
- L23:784 suppresses
- L23:489 suppresses
- L22:2580 suppresses
- L21:2248 suppresses
- L25:454 excites
- L22:311 excites
- L24:1528 suppresses
- L22:2309 suppresses
- L25:2691 suppresses
- L20:1731 suppresses
- L18:2722 suppresses
- L17:2209 excites
- L25:409 suppresses
- L22:201 weakly excites
- L16:1139 weakly suppresses 'and' in lists of places or names
- L22:1259 weakly suppresses
- L21:2160 weakly excites
- L20:3010 weakly suppresses
- L16:2649 weakly suppresses
Feeds later layers
- L27:1233 suppresses
- L27:1593 weakly suppresses
- L27:922 weakly suppresses
- L27:2774 weakly suppresses
- L27:759 weakly excites
- L27:385 weakly suppresses
- L27:1677 weakly suppresses
- L27:1458 weakly suppresses
- L27:942 weakly excites
- L27:694 weakly suppresses
- L27:2111 weakly excites
- L27:808 weakly excites
- L27:1388 weakly excites
- L27:1364 weakly suppresses
- L27:979 weakly suppresses
- L27:1317 weakly suppresses
- L27:2017 weakly suppresses
- L27:168 weakly excites
- L27:1556 weakly excites
- L27:1061 weakly suppresses
- L27:788 weakly excites
- L27:1357 weakly suppresses
- L27:268 weakly suppresses
- L27:446 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- arsed
- onna
- ually
- either
- either
- EITHER
- gresql
- |#
- >|
- >{
Suppresses
- 等
- etc
- etc
- 等等
- 等内容
- 等方面
- 等领域
- 等地
- 等问题
- 等因素