first token of a text

What this neuron does

This neuron fires on the very first token of a piece of text when that token is a line break. It does not respond to the meaning of the words, only to the start of a document. Steering with it gave unrelated text.

Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).

Fires most on

The text windows where this neuron's activation was highest. Amber marks positive activation; the outlined token is the peak. Hover a token for its strength.

1. #1 peak

   ⏎1. n. Any one of numerous species of marine gastropods belonging to

2. #2 peak

   ⏎- SOA services have self-describing interfaces in platform-independent XML documents. Web

3. #3 peak

   ⏎Lin CC, Lin XM, Zou Y, Zhang L (200

4. #4 peak

   ⏎Some people use their fridge to store everything in their kitchen. Your fridge is not

5. #5 peak

   ⏎know much about it beyond that and couldn't find much information about⏎it on

6. #6 peak

   ⏎First of all, let us know what is Gandhi Jayanti? We can proudly

Steering

Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.

Most negative on

Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.

1. #1 peak

   <|endoftext|>House in Space, from a NASA Remix Challenge. Credit: Cookieater20

2. #2 peak

   <|endoftext|>10 Tips for Parents of Prospective College Students⏎1. Choosing a Career

3. #3 peak

   <|endoftext|>Active transportation refers to non-motorized transportation modes, such as bicycling and

4. #4 peak

   <|endoftext|>Public holidays in Pakistan are different than bank holidays in the country. While all of

Which neurons in earlier layers feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.

Activation histogram

How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.

Logit lens

The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layers instead.