"that" pointing back to a noun

What this neuron does

This neuron fires on the word "that" when it points back to something just mentioned, as in "for that product" or "at that time". Turning it up made the writing wander without a clear link to this pattern.

Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).

Fires most on

The text windows where this neuron's activation was highest. Amber marks positive activation; the outlined token is the peak. Hover a token for its strength.

1. #1 peak

   On June 26, the Chinese observed "an extraordinary star" that was described as greyish-yellow, and was "as large as a pellet." On that same date, David Rittenhouse (Norriton, Pennsylvania, USA)

2. #2 peak

   “tracing request” to restore links with family members across the border via that platform.⏎[F5] The following day State A’s forces commence arrests of people

3. #3 peak

   , and the effect may be determined only through the analysis of process establishment data for that product.⏎Initial temperature is normally determined using a dial, electronic or other type of

4. #4 peak

   , while on the other, individual teams and firms can track the development of open source software through tools created for that purpose. There is a multitude that is available on the market, many of which

5. #5 peak

   on September 10, then received the name of Hurricane Humberto at the first report of the National Hurricane Center (NHC) the next day. At that time the hurricane was located about 310 mi (500 km

6. #6 peak

   bars denote the size of the inflation change in each item relative to its historical norm, measured by the standard deviation of the seven-month change in the inflation series for that item since 2000. (We go back only to 2

Steering

Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.

Most negative on

Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.

1. #1 peak

   ia in human culture⏎In the last quarter of the 18th century, Professor Jacob Seydelmann of Dresden developed a process to extract and produce a more concentrated form for use

2. #2 peak

   the university. The Female Law Practitioners Act had been passed in 1896, and on 10 May 1897 Ethel Benjamin gave effect to the new legislation when she was admitted as a barrister and

3. #3 peak

   that it competed with the figural component for the viewer’s attention, and eventually landscape developed into an independent genre. The paintings of German artist Albrecht Altdorfer mark a significant step in this evolution.⏎The majority of Altdor

4. #4 peak

   92 she passed the university junior scholarship examination.⏎In 1893 Ethel Benjamin enrolled for an LLB degree at the University of Otago. She

Which neurons in earlier layers feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.

Activation histogram

How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.

Logit lens

The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layers instead.