multiplication and scaling in math

What this neuron does

This neuron responds to math text where a number is multiplied or scaled, such as 'times 5', 'x 100', or 'multiplied by 2 raised to'. It goes strongly negative on the 'log' in decibel formulas. Turning it up made the model's writing repetitive and did not produce clear math.

Written by Claude from this neuron's activations and steering outputs. Confidence: low. Think it is wrong? Flag it (opens a prefilled issue on GitHub).

Fires most on

The text windows where this neuron's activation was highest. Amber marks positive activation; the outlined token is the peak. Hover a token for its strength.

1. #1 peak

   5x dx,⏎- but we know how to solve that. We can multiply by 5 and divide by 5.⏎- 1/5 times 5 is just going to be 1. So we haven't changed the value

2. #2 peak

   2.4 \times 100 \div 3.2$.⏎You could, of course

3. #3 peak

   int exp);⏎float ldexpf(float x, int exp);⏎long double ldexpl(long double x, int exp);⏎These functions computes the quantity x * 2exp.⏎Upon successful completion, these functions return x multiplied by 2 raised to

4. #4 peak

   ( 15 pF or 33 pF ) may be selected.⏎The tree positions of

5. #5 peak

   Gradient descent takes small steps to reduce the loss of a model. Gradient descent algorithms multiply the gradient by a scalar known as the learning rate (also sometimes called step size) to determine the next point. For example, if the gradient magnitude is

6. #6 peak

   is selective (varying through the spectrum). The distortions magnify the original factor 1 + d / to a factor that approaches 1 + 3d / . More precisely, the photographic magnitudes, to use the technical measure of

Steering

Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.

Most negative on

Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.

1. #1 peak

   be MAX.⏎Express the PSNR in decibels. From Step 1, we have the decibel value LdB as LdB = 10 log10 (P1/P0). Now let P1 = MAX^2

2. #2 peak

   the probability is about cxn, where c = (10/9)2 ≈ 1.2 and x = (9/10)log102 ≈ 0.97. The expected number of powers

3. #3 peak

   are in the same units of measure. The decibel is 0.1 bel, so the decibel value LdB is LdB = 10 log10 (P1/P0).⏎Define the mean squared error (MSE

4. #4 peak

   10 log10(MAX^2/MSE) = 10 log10(MAX/(MSE)^(1/2))^2 = 2

Which neurons in earlier layers feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.

Activation histogram

How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.

Logit lens

The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layers instead.