'scripting' before 'language', language loops
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 25 of Qwen3-0.6B. Its strongest snippets are fairly consistent: several of them peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron is hard to pin down. Its strongest negative reactions come on the word 'scripting' in phrases like 'server-side scripting language'. Turning it down made the model get stuck repeating the word 'language', so the neuron seems tied to the word 'language' in programming contexts.
Written by Claude from this neuron's activations and steering outputs. Confidence: low. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
ְדָה (vÿ’ikkavÿda), the Niphal cohortative; coming after the perfect tenses with vav (ו) consecutives expressing the future, this cohort
-
to tropical storm status, with maximum sustained winds of 65 mph (105 km/h).⏎On September 14, Humberto had reached post-tropical cyclone status, with winds at 40 mph (65 km
-
are very cheap and reliable in cost when we compared with the OP-Amp applications in the same areas. These ICs are used as an astable and monost
-
>> Project. Select General >> Empty Project and type in the project name.⏎- Under folder Source, right-click Select Add >> New Item.⏎- Under Code, Select C++ File (.cpp), however, when you type in the name use a
-
79. The withdrawal of RBGAN occurred because it was superseded by the newer BGAN technology.⏎BGAN signal acquisition requires line-of-site with the geostationary satellite and requires the user to have a compass and a general idea of
-
⏎- Cancer (malignant neoplasms)⏎- Chronic lower respiratory disease⏎- Accidents (unintentional injuries)⏎- Stroke (cerebrovascular diseases)⏎- Alzheimer’s disease⏎- Influenza and pneumonia
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: language and language language language language language language... language language language language language language language language language language language... language language language language language language language language language language language language... language language language language language language language language language language language language language...
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
the most widely used server-side scripting language⏎At time of this writing, PHP is the most used server-side language
-
, nearly every databases – MySQL, ORACLE, PostgreSQL, Sybase, Informix to name a few.⏎What is PHP?⏎- PHP is a server-side scripting language that means its code is executed on
-
although there are other possibilities. ⏎Students are expected to develop their understanding of programming beyond the skill level (that is, beyond the particular commands and syntax of a specific language) and develop an understanding of the general principles and characteristics of programming and programming
-
features is essential. This helps developers make informed decisions regarding the choice of programming languages, frameworks, and development tools. Secondly, adhering to coding best practices
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L23:443 suppresses
- L21:652 suppresses
- L15:921 excites
- L20:1634 suppresses
- L21:3028 suppresses
- L21:269 excites
- L19:552 suppresses
- L6:2790 excites
- L21:1206 excites
- L8:67 weakly suppresses
- L7:2300 weakly excites
- L23:2267 weakly excites
- L24:851 weakly suppresses
- L9:49 weakly suppresses
- L9:411 weakly excites
- L9:2711 weakly excites
- L5:135 weakly suppresses
- L4:1128 weakly suppresses
- L8:2765 weakly excites
- L7:2721 weakly excites
- L20:429 weakly excites
- L7:833 weakly excites
- L9:2033 weakly excites
- L21:1793 weakly suppresses
Feeds later layers
- L27:1855 strongly suppresses
- L26:523 weakly suppresses
- L27:908 weakly excites
- L26:1427 weakly excites
- L26:3036 weakly excites
- L27:2840 weakly suppresses
- L27:2147 weakly excites
- L27:2553 weakly suppresses capital T starting a word
- L26:1744 weakly suppresses
- L27:1216 weakly suppresses
- L26:2385 weakly excites
- L26:2568 weakly excites
- L26:2948 weakly excites
- L27:2508 weakly excites
- L26:1812 weakly excites
- L27:1473 weakly excites
- L26:1599 weakly excites
- L27:2979 weakly excites
- L26:3028 weakly suppresses
- L26:1196 weakly excites
- L27:1448 weakly suppresses
- L27:629 weakly excites
- L26:1136 weakly excites
- L27:834 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- software
- ASTER
- 软件
- resign
- класс
- akte
- uty
- umer
- software
- {*
Suppresses
- 语言
- language
- Language
- languages
- language
- Language
- 的语言
- 語言
- _language
- lang