Single features under the microscope

Members — a guided walk

1. Dictionary feature

   A dictionary feature is a single learned row of the sparse autoencoder's weight matrix M -- equivalently, one hidden unit of the autoencoder -- representing a direction in activation space used to reconstruct model activations as a sparse linear combination. Cunningham et al. (2023) treat individual dictionary features as their central unit of analysis, examining them in turn for interpretability, causal importance, and monosemanticity. This theme follows two such features under close inspection to see whether monosemanticity, examined from the input side, the output side, and a feature's causal neighbors, holds up as more than an aggregate score.

2. Monosemanticity

   Monosemanticity is the property of a feature activating for a single, coherent, human-interpretable concept, the converse of polysemanticity. Section 5 of the paper tests it with three analyses of individual dictionary features: which text activates a feature (input), how ablating it changes the model's output logits (output), and which upstream features cause it to activate (intermediate). The first case study, an apostrophe-detecting feature, walks through the first two of these directly.

3. Apostrophe dictionary feature (feature 556)

   Dictionary feature 556 activates almost exclusively on apostrophe tokens: a histogram of the tokens that trigger it shows the majority of high activations are apostrophes, with lower activations concentrated on conceptually similar punctuation. This contrasts with the corresponding default-basis residual-stream dimension -- the 10th-highest dimension the feature itself reads from -- which is polysemantic, representing different information at different activation ranges. On the output side, ablating feature 556 with less-than-rank-one ablation, which lowers the activation only as far as the feature's own threshold, mainly suppresses the logit for the following 's' token, exactly the effect expected of a detector used to predict the possessives and contractions like 'let's' that follow an apostrophe. Having read one feature end to end from input to output, the paper turns to a second feature and a third question: what causes a feature to fire in the first place.

4. Automatic feature circuit detection

   Automatic feature circuit detection answers that question for a chosen target feature: it samples contexts where the target strongly activates, ablates each dictionary feature in the previous layer in turn, and ranks those features by how much their removal lowers the target's activation, optionally recursing to build a multi-layer causal tree built entirely from ablations rather than inspection of weights. The method needs a target feature whose own meaning is already well established, so its clearest demonstration reuses the case-study logic of the apostrophe feature on a second feature, in layer 5, that predicts closing parentheses.

5. Closing-parenthesis dictionary feature

   The closing-parenthesis feature, dictionary feature 52027 in layer 5, is chosen because unembedding it directly predicts closing-parenthesis tokens, confirming its meaning independently before circuit detection is run at all. Tracing its upstream causes surfaces features detecting dates, acronyms, and other phrases that typically precede a closing parenthesis, an interpretable circuit assembled feature by feature. Between the apostrophe feature's input-output profile and this feature's upstream circuit, monosemanticity stops being a score reported in aggregate over thousands of features and becomes a property demonstrated concretely, one feature at a time.

Superthemes

• Decomposing the model into features

Edges explored within this theme

• Apostrophe dictionary feature (feature 556) — demonstrates the limits of → Monosemanticity
• Apostrophe dictionary feature (feature 556) — is benchmarked against → Default (neuron/residual-stream) basis baseline
• Closing-parenthesis dictionary feature — seeds → Automatic feature circuit detection
• Apostrophe dictionary feature (feature 556) — divides case study labor with → Closing-parenthesis dictionary feature

Super edges touching this theme

• Features proved by intervention — runs its proof on separate features than → Single features under the microscope

Sources

Every passage behind this theme’s member concepts.