"the" inside a prepositional phrase
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 16 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron lights up on the word "the" when it sits inside a phrase like "of the week," "at the same time" or "in the end," where a noun is about to follow. When it was turned up, the model fell into loops of "the" and "of the."
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
may be on call one day of the week or all week. Surgeons may work late-night or early-morning<|endoftext|>
-
of water (8 oz/240 mL).⏎Take Ampicillin on a regular schedule to get the most benefit from it. Taking Ampicillin at the same time each day will help you remember to take it.⏎To clear up your
-
(Session I)⏎Please note that presentation times are approximate. If you are interested in attending sessions with multiple presentations, please be in the room at the start of the session.<|endoftext|>Healthy Food on a Shoestring Budget
-
. This may lead to a difficulty in controlling your metabolism and a general slowing of the metabolism. The latter of these typically makes it much harder to lose weight in the end. Yoyo dieting may also put other strains on different parts of your
-
most common factors that cause a receding hairline and male pattern baldness are:⏎1. Genetically Caused – Sometimes hair loss occurs because it runs in the family. Yes, hair loss and baldness can be genetic. It was thought
-
adults focus on the value of being in the class and helping the class become a successful place of learning.⏎The instructor presented a
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: of all (or any) any of my such of the my kind of any of... of (I, or any of such, of the and... of the of (some) of the... of the of... of all of the
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
Numeral 1 in Roman Numerals converter tool is available here. This tool can prove to be very handy in the conversion of Arabic to Roman numerals. The tool basically works on a very simple concept of format conversion. In other words you
-
⏎When it comes to supplementing methionine in swine diets, the two primary sources are HMTBa (2-hydroxy-4-methyl
-
|⏎|Numbers||Mm in m||M in mm|⏎Some another meter to mm converter examples⏎The meters to millimeters converter is a great tool. The converter has a wide range of applications, both for individual and scientific applications. Often
-
13.5 out of 15 as a percent, you’d enter a 13.5 out of 15 into a calculator. The calculator will do the rest, including converting the decimal to a percentage.⏎To find
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L9:24 strongly excites "the" before a named organisation or title
- L4:52 strongly excites
- L3:291 excites
- L0:26 suppresses
- L6:61 suppresses
- L1:59 excites 'the' after a verb or preposition
- L12:435 suppresses
- L10:120 suppresses
- L10:1377 suppresses
- L9:662 suppresses
- L1:2597 suppresses
- L14:169 suppresses
- L0:39 excites 'the' right after 'On' in 'On the other hand'
- L14:254 excites
- L0:774 suppresses
- L5:1172 excites
- L2:73 suppresses
- L15:78 excites
- L5:558 excites
- L11:1168 excites
- L6:13 excites
- L15:82 suppresses
- L15:136 suppresses
- L9:1811 weakly excites
Feeds later layers
- L17:30 strongly excites
- L18:641 strongly excites
- L20:361 strongly excites
- L17:2190 strongly excites 'the' starting an object phrase
- L26:1941 strongly excites
- L21:614 excites
- L18:2380 excites "the" after certain verbs and prepositions
- L19:427 excites
- L17:1481 excites 'the' after 'some of' or 'most of'
- L19:3016 excites
- L18:799 excites
- L18:3000 excites
- L19:1376 excites
- L18:1547 excites
- L22:182 excites
- L18:355 suppresses
- L17:1652 suppresses
- L17:1056 suppresses
- L17:2135 excites 'the' after 'of' and similar words
- L20:1246 excites
- L17:2970 excites
- L18:602 excites
- L19:2492 excites "one of" before a group
- L27:148 excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- oretical
- same
- latter
- ories
- same
- slightest
- resulting
- entire
- rest
- following
Suppresses
- mate
- themselves
- classmates
- 姹
- osterone
- 同事
- ende
- the
- 时段
- @author