optimisation and attractor terms

What this neuron does

This neuron responds to technical words about optimising or finding a best point, such as 'minimum of' a loss function, 'attractors' and 'particle swarm optimisation'. It is pushed below zero on words like 'mating' and 'learning rate' in the same kind of text. Turning it up made the model write about counting and snow puzzles, not optimisation.

Written by Claude from this neuron's activations and steering outputs. Confidence: medium. Think it is wrong? Flag it (opens a prefilled issue on GitHub).

Fires most on

The text windows where this neuron's activation was highest. Amber marks positive activation; the outlined token is the peak. Hover a token for its strength.

1. #1 peak

   group of the second order learning methods. It follows a quadratic approximation of the previous gradient step and the current gradient, which is expected to be close to the minimum of the loss function, under the assumption that the loss function is locally approximately square,

2. #2 peak

   situated at the valley of the mountain. These statues represent the style of Phnom Da. Among the five

3. #3 peak

   rules, and aesthetic graphics drawn from the universe of mathematics and art.⏎Chaos and Fractals is divided into six parts: Geometry and Nature; Attractors; Cellular Automata,

4. #4 peak

   A vacation on Mars - an artist's journey in a computer graphics world (C.A. Pickover). Part II. Attractors. Automatic generation of strange attractors (J.C. Sprott). Attractors with dueling symmetry

5. #5 peak

   and tertiary structure motifs.⏎Aim 2 : Create and validate an accurate and scalable approach for using experimental base pairing and through-space tertiary constraints to drive three-dimensional fold refinement for large RNAs.⏎Aim 3 : Integrate the technologies developed

6. #6 peak

   the cell has to get the right amino acids in the right order. The order of the amino acids has to be just right to give the protein the right three-dimensional shape and the right electric charge distribution to make it do a job.⏎Since the

Steering

Push this neuron up or down while the model writes, and watch what changes. The outputs were generated ahead of time, so nothing runs in your browser.

Most negative on

Where the activation went furthest below zero. Blue marks negative activation. Many neurons mean something different in each direction.

1. #1 peak

   basically an optimisation method that mimics evolution. You have a set of candidate solutions (called a population). The best solutions get combined with each other (mating) and also some of those get randomly mutated. This lets the algorithm both use

2. #2 peak

   satisfy its creditors, owners, and other providers of capital, or they will invest elsewhere.⏎The following is the WACC calculation formula:⏎WACC = E/V × Re + D/V × Rd × (1 - Tc)⏎Re = cost

3. #3 peak

   HyperParameters:⏎Gradient descent takes small steps to reduce the loss of a model. Gradient descent algorithms multiply the gradient by a scalar known as the learning rate (also sometimes called step size) to determine the next point. For example,

4. #4 peak

   aphazardly across the bottom of the well like a quantum mechanics experiment gone horribly wrong:⏎When Learning rate is just right:⏎EPOCHs is one of the HyperParameters that you can

Which neurons in earlier layers feed this one, and which later ones it feeds. Amber links excite, blue links suppress; longer bars are stronger.

Activation histogram

How often this neuron fires at each strength. Almost always it does close to nothing (the tall bars in the middle); the snippets above are the rare moments it fired hard, amber for positive and blue for negative.

Logit lens

The words this neuron nudges the model toward (amber) or away from (blue) when it fires. A hint, not a label: many neurons work through later layers instead.