comma inside large numbers

What this neuron does

This neuron fires on the comma in big numbers like 1,526,006 or 1,332, the one that splits thousands from the rest. Turning it up made the model's text drift into ordinary sentences and numbers, which tells us little about the neuron itself.

Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).

Fires most on

The text windows where this neuron's activation was highest. Amber marks positive activation; the outlined token is the peak. Hover a token for its strength.

1. #1 peak

   (250.2 km2)|⏎|• Water||0.7 sq mi (1.7 km2)|⏎|Elevation||1,148 ft (350 m)|⏎|Time zone||Eastern

2. #2 peak

   As of the 2010 Census, the city had a population of 1,526,006, growing to 1,547

3. #3 peak

   silver production were between 1872 and 1883. At the peak of its prosperity, the town had an estimated population of 6,000"⏎"

4. #4 peak

   Ireland. The most notable jewel in the crown is a pale yellow brilliant in the centre of the front cross, weighing four carats. The crown contains 1,332 other diamonds, as well as pearls edging the band. Some

5. #5 peak

   Mahadev Temple is located in Dhyansar village of Una district of Himachal Pradesh. It is situated at an altitude of about 1,500 meters above sea level and offers stunning views of the surrounding hills and

6. #6 peak

   1, in World War II, the German battleship Bismarck sank the British battle cruiser Hood in the North Atlantic, with the loss of 1,300 lives.⏎In 1975 President Gerald Ford approved more

Steering

Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.

Most negative on

Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.

1. #1 peak

   second voyage to the New World the French arrived off the coast of Florida on Thursday, June 22, 1564 around three or four o’clock in the afternoon. They landed about thirty leagues south of the St. Johns

2. #2 peak

   nine. 3. My friend (to ring) me up at eight o’clock tomorrow. 4. Where Kate (to go) when you (to

3. #3 peak

   it’s been planted. Watering slowly and deeply is better for your tree’s over-all health then fast and frequent waterings. Watering before 10 a.m. and after 8 p.m. can reduce the amount of water lost

4. #4 peak

   few fires but the drought hit us hard.⏎In the hills surrounding the town and further afield the evidence was obvious with many stands of trees brown and dying.⏎Although this

Which neurons in earlier layers feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.

Activation histogram

How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.

Logit lens

The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layers instead.