'since' after a change or growth
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 7 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the word 'since' when it comes after a sentence describing something that has changed or grown, like 'has improved since 2002' or 'nearly doubled since 2009'. It is one of the clearest patterns in this set. Steering the model with this neuron did not produce text that shows the same theme, though.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
newborn care. They then teach pregnant mothers and their families the simple but important actions they can take to preserve the lives of newborns.⏎Although general health has improved since 2002, Afghanistan remains a country with some of the poorest health
-
⏎October 30, 2019⏎Uncle Sam needs you, so long as you are male.⏎Women’s equality has drastically increased since the passing of the 19th amendment 99 years ago: divorce
-
. This screening has continued with students in Years 2 and 3.⏎Boondall State School 's Year 3 Literacy Test results have increased significantly since the ELF program began. The percentage of students identified as being in the bottom
-
the phrase has shifted since then, referring not to patrons and their clients, but 'a collective political preference
-
sentences. The UN Working Group on Enforced or Involuntary Disappearances (WGEID) stated that while Chile has made great strides since it transitioned to democracy, many challenges remain. While the WGEID praised
-
involving both prescribed and illicit opioids are rising sharply. They’ve nearly doubled since 2009, and have infiltrated all genders, demographics and geographic
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: How to calculate the angles of a regular polygon (n) of a regular polygon? Remembering the information from the problem, but without the actual data. We can assume that the regular polygon is given in the form (n). Given a
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
wax as lip balm.<|endoftext|>Health care employment has been the bright spot in the otherwise lackluster recent jobs reports. As overall employment decreased by 2 percent from 2000 to 2010, employment in the
-
waters provide excellent fishing grounds. Peru is the world's second largest producer of silver and copper.⏎The Peruvian economy grew by an average of 5.6% per year from 2009-13 with a stable exchange rate
-
to commercial use in the mid-1960s. Container use has continued to grow steadily in the United States and is projected to grow at 6 percent annually. Presently, 80 percent of U.S. overseas cargo trade
-
through various arts and crafts projects. Provide them with art<|endoftext|>Reverse transcriptase from Moloney murine leukemia virus (MMLVRT) is an RNA dependent DNA polymerase, which has been used as a fundamental tool for molecular biology and biotechnology
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L6:2539 suppresses
- L5:2919 excites
- L6:1732 excites
- L6:1452 suppresses
- L5:685 suppresses
- L4:1290 excites
- L6:1890 suppresses
- L4:2220 weakly excites
- L4:1483 weakly excites
- L6:2052 weakly suppresses
- L1:2641 weakly suppresses
- L5:198 weakly excites unclear: weak "and" in lists
- L6:2979 weakly suppresses
- L6:801 weakly suppresses
- L3:2315 weakly suppresses
- L6:1282 weakly suppresses
- L4:605 weakly excites
- L3:2898 weakly suppresses
- L6:2665 weakly suppresses
- L6:2044 weakly suppresses
- L4:266 weakly excites
- L5:1691 weakly suppresses
- L1:3033 weakly suppresses
- L5:1462 weakly suppresses
Feeds later layers
- L14:867 excites
- L12:2924 excites
- L19:1499 excites
- L9:743 excites
- L11:1895 excites
- L9:2603 excites
- L17:1071 excites
- L11:239 suppresses
- L22:1555 excites
- L8:2196 excites comparison words like more, same, but
- L8:1732 excites
- L12:1177 excites
- L12:2918 excites
- L11:1755 excites
- L21:1734 excites
- L12:1122 excites
- L13:2961 excites
- L8:3049 excites
- L17:2739 suppresses
- L9:2310 weakly excites
- L9:803 weakly excites
- L23:477 weakly excites
- L10:2139 weakly excites
- L16:1678 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- differences
- changes
- _changes
- _changed
- 的变化
- difference
- 变化
- difference
- 改变
- 変わって
Suppresses
- ную
- айн
- очный
- 股市
- ydı
- GLISH
- ][/
- 科学院
- aviest
- administering