'of' inside set phrases like 'course of'
A neuronOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in layerOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. 21 of Qwen3-0.6B. Its strongest snippets are very consistent: they mostly peak on the same token. It is featuredA neuron picked for the full treatment: steering outputs and a written explanation. About 3,000 of the 86,000 neurons are featured., so it has steering outputs.
What this neuron does
This neuron lights up on the word 'of' in fixed phrases such as 'course of action', 'one of the largest' and 'Table of Contents'. It goes quiet or flips negative on 'of' in 'of course', and turning it up made the model drift into odd 'of' and 'the of' loops.
Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).
Fires most on
The text windows where this neuron's activationThe number a neuron produces for a token. Large positive means it fired hard; negative means it pushed the other way. was highest. Amber marks positive activation; the outlined tokenA piece of text the model reads at once: often a word, sometimes part of a word or a punctuation mark. is the peak. Hover a token for its strength.
-
by University of Minnesota students. You can see all of the details involved in making it in the video below.<|endoftext|>Create a Graph⏎Library Home || Full Table of Contents || Suggest a Link || Library Help⏎|National Center for Education Statistics
-
grammatical error in this sentence. It should be punctuated with a question mark, not a period. Second, the title of fourth-grade social studies’ course of study is North Carolina History. This question does not address
-
about virtuous living by thinking over the qualities of each virtue. 2. Meditation can help a child learn to think for themselves, and determine a best course of action by reflecting on possible solutions. 3. If the child is being raised
-
of the state of Oaxaca. He was president of Mexico five times. He helped draft the Plan of Ayutla, demanding the deposition of Santa Anna and the implementation the Constitution of
-
is applicable. In general, these models are applied to problems with the following characteristics:⏎The problem has a well-defined sole objective.⏎There are alternative courses of action to resolve the problem.⏎The decision variables are continuous and they can accept any
-
Achievement Gap⏎Each year more than 10,000 youth in Minnesota disengage from school, and Minnesota’s academic achievement gap has grown to one of the largest in the nation.⏎Have you considered leaving school? Are you currently dis
SteeringAdding a push along a neuron while the model writes, to see how its output changes. Turning it up or down.
Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.
Here is something I have been thinking about lately: course of course, course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course course
Most negative on
Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.
-
opportunities or even just advice. They can connect⏎directly with employers and recruiters who can help them reach their goals faster than they ever thought possible. And, of course, Guru99 notes that online job boards function as the perfect resource for
-
strap on our virtual reality goggles, and hit the weights.⏎Researchers from the University of Wisconsin-Madison are doing something like that (although, of course, the weights don't weigh anything) by creating a virtual reality simulator called
-
Airplane” was once quoted as saying “If you can remember anything about the 60’s, then you weren’t really there!!”. He was, of course, referring to the rampant drug use which has become intrinsically associated with
-
answered or increasingly misplace items -- forget appointments, forget medications. If these are new changes, then that's a worry sometimes for Alzheimer's disease. And, of course, if this forgetfulness increases, then other people have to help the individual
Connections
Show on mapWhich neuronsOne of thousands of tiny dials inside each layer. It reads the text so far and produces a number, its activation, that nudges what the model writes next. in earlier layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.
Fed by earlier layers
- L19:715 strongly excites
- L20:2800 excites
- L20:2860 excites
- L17:317 excites 'of' after a noun phrase
- L20:1917 excites
- L20:1356 excites
- L20:2599 suppresses
- L19:2507 suppresses
- L20:2962 suppresses "of" in "most X of all" superlatives
- L19:2128 excites
- L19:2622 excites
- L19:1503 excites
- L16:961 excites
- L18:1954 suppresses
- L19:3054 excites
- L19:650 weakly suppresses
- L18:769 weakly excites
- L17:1771 weakly excites
- L17:2409 weakly suppresses
- L20:55 weakly suppresses
- L7:656 weakly excites
- L20:302 weakly excites
- L18:445 weakly excites
- L1:1940 weakly excites
Feeds later layers
- L26:711 strongly suppresses 'years' after an age number
- L23:567 strongly suppresses
- L22:142 strongly excites 'of' after a quantity or noun phrase
- L27:447 strongly suppresses
- L23:258 strongly suppresses
- L25:2020 strongly suppresses
- L26:115 strongly excites
- L26:920 strongly suppresses
- L22:288 strongly suppresses
- L24:860 suppresses
- L26:279 excites
- L25:18 excites
- L27:658 suppresses
- L22:1993 suppresses
- L24:2753 excites
- L23:816 suppresses
- L23:2665 suppresses
- L24:1558 suppresses
- L26:3056 suppresses
- L26:309 excites
- L27:1582 weakly excites
- L24:359 weakly suppresses
- L26:1320 weakly suppresses
- L26:54 weakly suppresses
Activation histogramA chart of how often a neuron takes each activation value.
How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.
Logit lensReading which output tokens a neuron pushes toward or away from, straight off its output weights, before the later layers have their say.
The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layersOne step in the stack the text passes through. Qwen3-0.6B has 28 layers, each with 3,072 neurons. instead.
Promotes
- знания
- issant
- 불구하고
- gend
- 什么地方
- zac
- יסט
- 并且
- 并与
- 并将
Suppresses
- of
- course
- course
- _of
- →of
- Of
- of
- Of
- của
- -course