Apostrophe dictionary feature (feature 556)
Sparse Autoencoders Find Highly Interpretable Features in Language Models — introduced
A case-study dictionary feature that activates almost exclusively on apostrophe tokens, the paper's central worked example of monosemanticity. Ablating it specifically suppresses the logit for the following 's' token, illustrating that dictionary features have intuitive, predictable causal effects on model outputs, in contrast to the corresponding polysemantic default-basis dimension.
A single learned feature carries an outsized share of this paper's case for interpretability: it's examined from every angle available, what text makes it fire and what happens to the model's output when it's removed, rather than getting only half the treatment most examples do. It turns out to be reassuringly sensible on both counts — mostly triggered by apostrophes, mostly responsible for boosting the letter that follows them in contractions.
The page covers what activates the feature and what it causally does downstream, the finding that this single human concept actually splits across several narrower features rather than living in just one, and how the case study divides its labor with a second feature covering what causes a feature to fire in the first place.
The paper's flagship demonstration of monosemanticity
The apostrophe feature, feature 556, is the central worked example in single-features-under-the-microscope: it is the one feature the paper subjects to both halves of Section 5's input/output analysis, making it the strongest single piece of evidence that a dictionary feature can be both interpretable from what activates it and causally predictable in what it does downstream.
Input: what activates it
Plotting a histogram of which tokens activate feature 556 across every datapoint where it fires shows the large majority of activations are apostrophes, especially at high activation strength; lower-activating tokens are still conceptually related, mostly other punctuation (sparse-autoencoders, §"5.1 Input: dictionary features are highly monosemantic", p. 7). This token-histogram technique only works for features that activate on a small, identifiable set of tokens, which is itself part of the point: the apostrophe feature is narrow enough to check this way at all, unlike a more diffuse polysemantic direction.
Output: what it causally does to the logits
Applying Less-than-rank-one ablation to the feature -- lowering its residual-stream contribution only down to the point where it stops firing -- shows that removing it mainly suppresses the logit for the following "s" token, matching the intuitive story that the feature detects apostrophes and helps the model predict the "s" that follows in possessives and contractions like "let's" (sparse-autoencoders, §"5.2 Output: dictionary features have intuitive effects on the logits", p. 7). This stands in contrast to the corresponding default-basis dimension most associated with apostrophes: the paper's own search finds this raw-coordinate dimension is polysemantic, since it "only explains a very small fraction of the variance for middle-to-lower activation ranges" where the dictionary feature stays clean (sparse-autoencoders, §"D.1 Residual stream basis", p. 16).
Not one apostrophe feature, but several
The feature does not activate on all apostrophes. Two further dictionary features fire on apostrophes too, but in different grammatical contexts -- one for contractions like "[I/We/They]'ll" and another for "[don/won/wouldn]'t" (sparse-autoencoders, §"5.1 Input: dictionary features are highly monosemantic", p. 7). What looks, from outside the model, like a single human concept of "apostrophe" is split, in the actual learned dictionary, across at least three narrower features, each tied to a distinct surrounding pattern -- evidence about the granularity at which Monosemanticity actually holds, not just that it holds.
Dividing the case-study labor with the closing-parenthesis feature
Section 5's three-part methodology -- input, output, and intermediate-feature analysis -- is not run in full on any single feature; the paper splits it across two case studies. The apostrophe feature carries both the input analysis and the output analysis, while the closing-parenthesis feature is reserved for the third leg, circuit tracing, because its final-layer position makes its own meaning nearly given for free. Together the two examples argue for two separate claims: that a feature is monosemantic by both what activates it and what it does, and that dictionary features compose into traceable circuits.