prepositions after comparison words
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 14 of Qwen3-0.6B. Its strongest snippets are fairly consistent: several of them peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron fires on the small word that follows a comparison, such as 'to' in 'compare to', 'from' in 'indistinguishable from', or 'than' in 'less than'. Turning it up did not bring out this theme in the generated text, so the snippets are the clearest evidence.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
REC ITY OSC⏎ALL IAT SQU⏎See another brain teaser just like this one...⏎Or, just get a random brain teaser⏎If you become a
-
tour of this brave new theory at the frontier of physics, showing what some of these alternate realities might be like. Some universes may be almost indistinguishable from our own; others may contain variations of all of us, where we exist but
-
day, I am told to put my fingers firmly in my ears. Even then, when the four quarters are struck, the noise is phenomenal, but nothing compared to the striking of the hour bell, Big Ben. The belfry reverber
-
. The artistic atmosphere and community in America could not compare to the one in Europe. According to H.L.W. Dana, „Washington All
-
on Earth is not a claim for Chinese exceptionalism, indeed quite the opposite. Almost all the 20 essential ideas it identifies are the same or very similar to
-
, while at the same time can be accessed in a uniform manner by all languages that the .NET platform support. In other words, CLS is a subset of the full functionality of CTS.⏎- Common Language Run-time (CLR):⏎The
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: the idea that the idea of the name "Carnegie" and the relationship between the two, the relationship between the two people, the name "Carnegie" and the 19th century, the relationship between the two,
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
their different characteristics.⏎Two word system developed by Carolous Linnaeus to name species, first word identifies the genus of the organism, the second word is often a descriptive word that describes a characteristic of the organism.⏎Next large taxon in the
-
in the following sequence:⏎moral, friend, port, feet, race⏎AnswerBy adding the sound of the first five letters of the alphabet to the beginning of each word respectively, another word can be enunc
-
function is equal to a variable:⏎- Your function name can be designed using characters from alphanumeric and an underscore.⏎- Your function name does not have to start with a number⏎- Your function name has to be different from an existing function or language
-
the Union...⏎7th - 12th Social Studies & History⏎President Obama's State of the Union Address 2011⏎Students pretend they are journalists and have to write 10 questions to ask the President at
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L11:2292 excites 'than' in comparisons
- L12:981 suppresses
- L6:984 suppresses
- L13:1944 suppresses
- L10:1465 suppresses
- L12:1945 excites
- L12:1197 suppresses
- L13:2434 weakly excites
- L12:1148 weakly suppresses
- L4:2034 weakly suppresses
- L13:1769 weakly excites
- L12:2692 weakly suppresses
- L12:2855 weakly excites
- L5:1317 weakly suppresses 'a' right after 'being'
- L10:1447 weakly suppresses
- L12:1496 weakly excites
- L5:872 weakly suppresses 'or' introducing an alternate name
- L11:1605 weakly suppresses
- L12:2298 weakly excites
- L6:1971 weakly suppresses
- L9:573 weakly excites
- L12:2930 weakly excites "and" joining contrasting pairs
- L9:1802 weakly suppresses
- L13:2356 weakly excites
Feeds later layers
- L16:963 suppresses
- L16:1181 excites
- L19:709 excites
- L16:2315 excites
- L16:2600 excites
- L16:1147 excites
- L16:935 suppresses
- L15:2006 excites
- L16:2521 excites
- L16:2039 weakly excites
- L18:2557 weakly excites
- L15:203 weakly excites
- L18:1039 weakly excites
- L16:101 weakly excites
- L19:2091 weakly excites
- L16:1045 weakly suppresses
- L18:1037 weakly suppresses
- L16:1959 weakly excites
- L16:2836 weakly excites
- L17:2631 weakly excites
- L15:1255 weakly excites
- L16:1913 weakly suppresses
- L20:1029 weakly excites
- L15:1145 weakly excites
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- regar
- vous
- erreur
- ']")⏎
- ANNOT
- .Default
- 为了
- aptive
- ateur
- alled
Suppresses
- latlong
- 榧
- Piano
- gie
- 联
- 程
- 毀
- _relationship
- Tone
- мир