function words before an expected phrase
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 16 of Qwen3-0.6B. Its strongest snippets are mixed: they peak on different tokens. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on small connecting words that set up a phrase still to come, like 'of' in 'think of', 'on' in 'based on', or 'the' after 'took'. Turning it up made the model's text wander, with no clear theme.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
antages and disadvantages of using Sunpower solar panels⏎The use of solar energy is becoming increasingly popular across the globe as the technology continues to develop and prices drop. One of the most popular types of solar energy is Sunpower solar panels. Sunpower
-
separation anxiety. When left alone for prolonged periods of time, dogs will begin to feel lonely and frightened. This can cause them to act out in many ways, including
-
in the garment industry has transformed cultural expectations of women and girls.<|endoftext|>Microcontroller vs FPGA: Understanding the Key Differences⏎When it comes to embedded systems design, two technologies that often come up are microcontrollers and field-programmable gate arrays (
-
24 hours.⏎3D printed valves⏎Once the Brescia hospital realised its regular valve supplier could not produce the valves on time, it was forced to think of alternative solutions as it had run out of breathing tubes for the intensive care machine.
-
that supposedly must have existed. Evolutionists needed evidence to back up the supposed transition of vertebrates from the sea to dry land. For that reason, they took the fossil coelacanth, whose anatomy they believed was ideally suited to this scenario
-
worldwide,” Waring says. “To combat global warming, we’re understandably going to make our buildings more sustainable. But in that process, we need to think of sustainability as both reducing energy demand and protecting the health of the building occupants.”⏎W
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the reason for the difference between the current time, the current time, and the difference between the current time. This is what you are saying, but I think the problem with this, the difference between the current time and the current time,
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
by By Dr. Carolyn K. Stratz⏎Q: My kids get their vaccinations - what about me?⏎A: Back-to-school time doesn't just mean immunizations for
-
was found in River Rio Cristalino in the southern Amazonas. It's nearest relative in the aquaristic is Hygrophila but Staurogyne is different with its marked compact, low and bushy stature (5-1
-
to come,” but “this is not a disease that the general public should be worried about. It’s not Covid or other fast-spreading diseases.” Monkeypox belongs to the same family as smallpox, which killed millions of people
-
. Other tools on the Voyant site are Word links (creates a mind map of words) and a scatter plot (for word frequencies). http://voyant-tools.org/⏎Wordle: Wordle is a free web<|endoftext|>His wife once threatened
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L15:245 suppresses
- L13:2538 excites
- L15:2963 excites
- L15:327 excites
- L14:2957 suppresses
- L13:801 suppresses
- L13:298 suppresses
- L13:1018 suppresses
- L13:1611 suppresses
- L14:219 excites
- L15:2476 suppresses
- L13:1347 excites
- L15:707 suppresses
- L15:1734 excites
- L15:2577 suppresses
- L14:2069 suppresses
- L13:2312 excites
- L13:1910 excites
- L13:1407 suppresses
- L15:159 suppresses
- L15:2641 excites
- L15:268 suppresses
- L7:2470 suppresses
- L14:891 excites
Feeds later layers
- L17:2175 excites
- L17:1687 excites
- L17:2094 suppresses
- L17:1151 excites
- L17:1624 excites
- L17:2851 excites
- L17:1098 suppresses
- L17:1329 excites
- L17:655 weakly excites
- L19:431 weakly excites
- L18:2291 weakly suppresses
- L17:2173 weakly suppresses
- L23:1339 weakly excites
- L19:1916 weakly excites
- L17:1915 weakly excites
- L17:1975 weakly excites
- L18:1886 weakly suppresses
- L19:823 weakly excites
- L17:1124 weakly suppresses
- L17:1240 weakly excites
- L17:1510 weakly suppresses
- L18:941 weakly suppresses
- L24:470 weakly suppresses
- L18:2275 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- rolled
- acular
- 山区
- 爱人
- acles
- 旅行
- avorite
- コンテンツ
- ipur
- @PostMapping
Suppresses
- kicking
- flu
- grub
- arm
- redirects
- obao
- retaining
- BLE
- epam
- grabbing