unclear: scattered mid-sentence tokens

What this neuron does

This neuron fires on a grab bag of ordinary words in the middle of sentences, such as 'at', 'the' and 'is', with no clear shared meaning. Turning it up made the model get stuck repeating a single word like 'She' or 'The'.

Written by Claude from this neuron's activations and steering outputs. Confidence: low. Think it is wrong? Flag it (opens a prefilled issue on GitHub).

Fires most on

The text windows where this neuron's activation was highest. Amber marks positive activation; the outlined token is the peak. Hover a token for its strength.

1. #1 peak

   station, municipal yard,⏎highway department, or school/library. There are some sites that⏎are operated by media such as newspapers, and a few that are at⏎ahead can help you locate just exactly where the station might⏎be

2. #2 peak

   ⏎Minerals can be identified by their hardness,⏎colour, the way they reflect light (lustre), the⏎way they break (cleavage and fracture), and⏎Mineral crystals fall into six crystal systems,⏎each of which is

3. #3 peak

   that it⏎bridges them. Using a straw, blow air underneath⏎the paper. This causes a downward force on the paper, and it⏎moves down in the center.⏎By blowing the air between

4. #4 peak

   Kansas, Texas, New Mexico, Montana, Alaska, and California; Canada=⏎Manitoba, Alberta, British Columbia; Mexico; Brazil; Poland; Denmark=Bornholm,⏎Greenland; Austria; Switzerland; Hungary;

5. #5 peak

   shapes in this example, labelled 1, 2 and 3 as follows:⏎2222.2222 ..22.2... 11.222..3 ....2..33⏎

6. #6 peak

   .⏎The surface is level or slightly undulating near the lake, but in the E.⏎part rough and mountainous, and more adapted to grazing. The soil is fertile,⏎particularly along the lake shore. Indian corn, wool, oats, and

Steering

Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.

Most negative on

Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.

1. #1 peak

   historical State of the Union Address to more contemporary ones. In this Government instructional activity, students compare and contrast historical Presidential speeches to the ones they hear today. High...⏎9th - 12th Social Studies & History⏎Anticipating the

2. #2 peak

   ons⏎together? How will you know you have found all the ways?⏎Swap the stars with the moons, using only knights' moves (as on a⏎chess board). What is the smallest number of moves possible?⏎Can you work

3. #3 peak

   this⏎Dig into the entry’s quotation paragraphs, and you’ll see that the OED

4. #4 peak

   02, 2012⏎Repairing bones with nanomaterials⏎(Nanowerk News) Around

Which neurons in earlier layers feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.

Activation histogram

How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.

Logit lens

The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layers instead.