Sparse Autoencoders Find Highly Interpretable Features in Language Models

A hypothesized cause of polysemanticity in which a network represents more features than it has dimensions by assigning them to an overcomplete set of non-orthogonal directions, relying on activation sparsity to limit interference between them. The paper's stated goal is to resolve superposition in language models directly, by recovering these directions via sparse dictionary learning rather than working around the problem.

The paper's opening problem: individual neurons activate in multiple, semantically unrelated contexts, preventing concise, human-understandable explanations of what a network is doing internally. Named in the abstract's first sentence as “one of the roadblocks to a better understanding of neural networks' internals,” it is the roadblock the paper's entire method is aimed at removing.

The property the method is judged by: a feature activating for a single, coherent, human-interpretable concept, the converse of polysemanticity. The paper's case studies — an apostrophe-detecting feature and a closing-parenthesis-predicting feature — are framed as demonstrating that learned dictionary features are highly monosemantic compared to neurons or other baseline directions, even if not every feature covers an entire human-nameable category on its own.

The research agenda the paper names as its home discipline: reverse-engineering a network's internal computations well enough to make targeted changes to them, motivated in part by the worry that AI systems might deceive humans to accomplish undesirable goals. The paper offers its method as “a foundation for future mechanistic interpretability work,” hoping it will “enable greater model transparency and steerability.”

The paper's method: a single-hidden-layer, ReLU, tied-weight autoencoder trained with an L1 sparsity penalty to reconstruct a language model's activations as a sparse linear combination of an overcomplete dictionary of learned features. The paper's central claim about this object is that the resulting features are more interpretable, more monosemantic, and more causally precise than neurons or any of four alternative decompositions tested.

A single learned row of the autoencoder's weight matrix, representing one direction in activation space used to reconstruct inputs as part of a sparse linear combination. Individual dictionary features are the paper's central unit of analysis throughout — examined for interpretability, causal importance, and monosemanticity in turn.

The L1 penalty on the autoencoder's hidden-layer coefficients, encouraging each activation to be reconstructed from as few active features as possible. The sparsity coefficient is the paper's key hyperparameter, swept throughout every experiment to trade sparsity against reconstruction accuracy rather than fixed at one value.

The squared-error term measuring how well the autoencoder's output matches the original activation it was given. The paper is explicit that this loss never reaches zero in its experiments — the dictionary does not capture everything a layer's activations carry, a gap the paper treats as an open limitation rather than something the method has yet resolved.

The hyperparameter controlling how overcomplete the learned dictionary is, the ratio between the autoencoder's hidden dimension and the model's own activation dimension. The paper finds interpretability gains hold even for undercomplete dictionaries, with larger dictionaries simply explaining more of a layer's overall activation variance rather than producing individually more interpretable features.

The paper's primary interpretability measure: a language model is shown text where a feature activates and writes a natural-language explanation, then a second model uses that explanation alone to predict the feature's activations on held-out text, with the correlation between predicted and actual activations as the score. Run across thousands of features, dictionary features score higher on average than every one of the paper's four baseline decompositions.

One of four baseline decompositions the paper's dictionary features are measured against. On the causal-localization test, PCA needs more patched components and larger edit magnitude than the dictionary to reach a given level of behavioral change — a gap the paper traces specifically to its sparsity penalty, since removing that penalty from the same architecture erases the advantage entirely.

The naive baseline of treating individual neurons, or individual residual-stream coordinates, as the unit of analysis. The paper's apostrophe case study makes the comparison direct: the best-matching raw coordinate the paper could find is still polysemantic across much of its activation range, where the matched dictionary feature stays clean throughout.

The paper's causal test: run the model on a counterfactual sentence, cache the resulting feature activations, then add the encoded differences for a chosen subset of features back into a base sentence's residual stream, measuring how far the output moves toward the counterfactual target. This reaches a given level of change using fewer patched features and a smaller edit magnitude than an equivalent patch built from PCA components.

The behavior used to test causal precision: a model correctly naming the recipient in sentences like “Alice and Bob went to the store. Alice gave a snack to ___.” Its circuit was already well-understood externally before this paper touched it, which is what lets the paper confirm that the features its patching procedure identifies as important are actually the ones responsible for the behavior.

A feature that activates almost exclusively on apostrophe tokens, the paper's central worked example of monosemanticity. Ablating it specifically suppresses the model's prediction of the following 's' token, giving it an intuitive, predictable effect on the model's output — in clear contrast to the polysemantic raw coordinate that shares its top activations.

A feature in the model's final layer whose unembedding directly predicts closing-parenthesis tokens — a meaning confirmable almost without any activation data at all, since a final-layer feature's unembedding directly names the tokens it promotes. Its near-unambiguous identity is what makes it a credible test case for the paper's circuit-tracing method.

A method for tracing which upstream features cause a target feature to activate: sample contexts where the target fires, ablate each candidate upstream feature in turn, and rank candidates by how much their removal lowers the target's activation. Applied to the closing-parenthesis feature, it surfaces upstream features detecting dates, acronyms, and other parenthesis-preceding phrases — an interpretable, sensible-looking circuit built from nothing but repeated ablation.

A check on how much information the dictionary actually loses: substitute a layer's activations with the autoencoder's reconstruction and measure the resulting rise in the model's perplexity. For one tested layer, this raises perplexity from 25 to 40, nearly doubling it — concrete evidence that a reconstruction loss which looks small in the abstract still costs the model real predictive accuracy.

An attempt to extend the method from the residual stream to MLP sublayer activations, reported as “mixed success”: many resulting features are more interpretable than individual neurons, but the approach suffers from large numbers of features that never activate at all, and does not yet reliably learn an overcomplete dictionary in the model's middle or later MLP layers.

The long-term goal the paper names in its conclusion: a complete, human-understandable enumeration of a model's features, sufficient to guarantee it will not perform dangerous behaviors such as deception. The paper is explicit that it offers only “a step towards” this ambition, not an instance of it — reconstruction never reaches zero, MLP dictionaries lose many features, and only a handful of layers on two small models have been tested.

Polysemanticity is named in the paper's opening sentence as the roadblock motivating everything that follows: neurons “appear to activate in multiple, semantically distinct contexts,” which “prevents us from identifying concise, human-understandable explanations for what neural networks are doing internally” (sparse-autoencoders, Abstract, p. 1). Using individual neurons as the unit of analysis “has had some success,” the paper concedes, but polysemanticity is the specific failure mode that success runs into.

Superposition is the hypothesized mechanism behind polysemanticity: a network represents more features than it has dimensions by assigning them to an overcomplete set of non-orthogonal directions, relying on sparse activation to keep interference between them manageable. “Without high sparsity, interference between non-orthogonal features prevents any performance gain from superposition” (sparse-autoencoders, §1 Introduction, p. 1) — the paper's stated goal is to resolve superposition by recovering those directions directly.

Monosemanticity is the converse property the paper's method is built to find: a feature or direction that activates for one coherent, human-interpretable concept rather than several unrelated ones. It becomes a demonstrated, not just scored, property once the apostrophe and closing-parenthesis case studies (Section 5) show individual dictionary features behaving this way under direct hand inspection.

Mechanistic interpretability is named as the field this method aims to advance: a research agenda seeking to “reverse engineer parts of [a network's] internal processes and make targeted changes to them,” motivated in part by the worry that AI systems “might deceive humans in order to accomplish undesirable goals” (sparse-autoencoders, §1 Introduction, p. 1). The paper positions its method as “a foundation for future mechanistic interpretability work.”

Sparse dictionary learning is the decades-old mathematical framework, developed by Olshausen & Field (1997) and Lee et al. (2006) to model sparse coding in the visual cortex, that the introduction identifies its problem with: finding features is “equivalent to the well-known problem of sparse dictionary learning.” The paper's sparse autoencoder is presented throughout as a scalable neural-network implementation of that older framework, applied here for the first time at this scale to language-model activations.

Sharkey, Braun & Millidge's 2023 unpublished interim report, titled “Taking features out of superposition with sparse autoencoders” — the exact phrase this paper's Section 2 heading borrows — first proposed training sparse autoencoders on language model activations and showed in toy settings that the idea could recover known ground-truth features. This paper, co-authored by Lee Sharkey himself, is explicitly framed as scaling that report up to a real language model and adding the autointerpretability, causal-localization, and monosemanticity evaluations it lacked.

A sparse autoencoder, in this paper's design, is a single-hidden-layer, ReLU, tied-weight network trained with an L1 penalty on its hidden activations to reconstruct a language model's activations as a sparse linear combination of an overcomplete dictionary. It is one of two founding empirical demonstrations, alongside the concurrent Anthropic “Towards Monosemanticity” work, that this architecture recovers features more interpretable and monosemantic than PCA, ICA, or the neuron basis (sparse-autoencoders, §2, p. 2).

A dictionary feature is one learned row of the autoencoder's weight matrix M — equivalently, one hidden unit — representing a direction in activation space used to reconstruct inputs as part of a sparse linear combination. It is the paper's central unit of analysis, examined for interpretability (Section 3), causal importance (Section 4), and monosemanticity (Section 5) (sparse-autoencoders, §2, p. 2; §5.1, p. 7).

The alpha·||c||_1 term in the training objective (Equation 4) penalizes the L1 norm of the hidden-layer coefficients c to encourage sparse feature activations. The sparsity coefficient alpha is the paper's key hyperparameter, swept throughout to trade sparsity against reconstruction accuracy, and is only meaningful because each dictionary feature is separately held to unit norm — otherwise the optimizer could game the penalty by rescaling M instead of genuinely zeroing out coefficients.

The ||x - x-hat||^2 term in the training objective (Equation 4) is the standard squared-error measure of how well the reconstructed activation matches the original. The paper is candid that this loss is never driven to zero, evidence the dictionary fails to capture all the information a layer's activations carry — a gap independently corroborated later by the reconstruction-substitution perplexity metric introduced as a limitation (sparse-autoencoders, §2, p. 3; §6.2, p. 8).

R controls the ratio of the autoencoder's hidden dimension to the model's activation dimension (d_hid = R x d_in), governing how overcomplete the learned dictionary is. Sweeping R shows interpretability gains hold even for undercomplete dictionaries (R<1), with larger dictionaries explaining more of the overall activation variance without any single feature scoring less interpretable on average (sparse-autoencoders, §2, p. 2; §C.1, p. 13).

Tied encoder/decoder weights — a single shared matrix M, transposed between encoder and decoder — halve the model's memory cost and remove any ambiguity about whether a feature's direction should be read from the encoder or the decoder, since with tying the two coincide by construction. The choice costs nothing measurable on residual-stream data, but hurts performance on MLP activations, motivating an untied variant there instead.

Sweeping the sparsity coefficient alpha traces a smooth curve trading the number of active features against unexplained reconstruction variance, with no distinct “knee” or single best operating point (sparse-autoencoders, §B, p. 12). The paper reads this as weak evidence against there being one uniquely correct sparse decomposition of a given activation space — every (alpha, R) setting is a defensible, merely different, decomposition.

GPT-4 is shown text where a feature activates and writes a natural-language explanation; GPT-3.5 then uses that explanation alone to predict the feature's activations on held-out text, and the correlation between predicted and actual activations is the score (sparse-autoencoders, §3.1, p. 3). Run across thousands of features and four baselines, this is the paper's primary measure of interpretability, adapted from Bills et al. (2023) and applied here at a scale no team of human annotators could match.

GPT-4 is the autointerpretability pipeline's explainer: shown 5 of a feature's 20 top-activating text fragments from OpenWebText, along with per-token activations rescaled to integers 0-10, it writes the single free-text explanation the rest of the pipeline treats as ground truth (sparse-autoencoders, §A, p. 11). Its explanations range from concrete (“the number 5... personal names and some nouns,” scored 0.57) to vague enough to anti-predict the feature's real behavior.

GPT-3.5 is the pipeline's simulator: given only GPT-4's finished explanation text, with no access to the original activating fragments, it predicts per-token activations on a fresh batch of fragments, and the score is the correlation with the feature's true activations. GPT-3.5 substitutes for GPT-4 in this role purely because OpenAI's public API exposed the logprobs the simulation step needs for GPT-3.5 but not, at the time, for GPT-4 (sparse-autoencoders, §A, p. 11).

PCA is one of four baseline decompositions the learned dictionary is measured against, fit on the same multi-million-vector activation dataset used to train the autoencoders themselves. On the IOI causal-localization test it underperforms the sparse dictionary noticeably, needing more patched components and larger edit magnitude to reach a given KL divergence — a gap the paper shows disappears if the sparsity penalty is removed (sparse-autoencoders, §3.2, p. 3; §4.2, p. 6).

ICA finds maximally non-Gaussian, statistically independent directions, and is consistently the strongest of the four non-dictionary-learning baselines — the sparse dictionary's autointerpretability advantage over it “declines as we move through the model, being comparable to ICA in layer 4 and showing minimal improvement in the final layer” (sparse-autoencoders, §3.2, p. 3). The paper attributes ICA's strength to interpretability correlating with activation kurtosis and skew, properties ICA directly optimizes for — though ICA is also trained on markedly less data than PCA.

Activation patching (Vig et al., 2020) is a causal mediation technique: run a model on a counterfactual input, cache its internal activations, transplant them into a run on the base input, and measure how far the output moves toward the counterfactual's. This paper adapts it to dictionary features — patching feature-encoded activation differences into the residual stream rather than whole raw activations — to test how precisely SAE features localize the IOI behavior (sparse-autoencoders, §4, p. 5; §4.1, p. 6).

The paper's specific procedure: run the model on a counterfactual sentence, cache the encoded feature activations, re-run on the base sentence, and add the encoded activation differences for a chosen feature subset into the residual stream, measuring KL divergence from the target output. It reaches a given KL divergence with fewer patched features and smaller edit magnitude than patching an equivalent number of PCA components (sparse-autoencoders, §4.1, p. 6; §4.2, p. 6).

Automated Circuit Discovery (Conmy et al., 2023) is a prior algorithm for finding the subgraph of a network responsible for a behavior. This paper reuses ACDC's ordering procedure — ablate each candidate feature and rank it by how much removing it moves the model's KL divergence from the target — to rank dictionary features by causal contribution, producing the feature subset used in the patching experiments (sparse-autoencoders, §4.1, p. 6).

The Indirect Object Identification task (Wang et al., 2022) is a previously-studied behavior in which a model completes sentences like “Alice and Bob went to the store. Alice gave a snack to ___” by naming the correct recipient. It serves as the paper's causal-localization testbed precisely because its circuit is already published externally, letting the paper confirm that features patching identifies as important actually are — a check the autointerpretability score has no way to supply (sparse-autoencoders, §4, p. 5).

Feature 556 activates almost exclusively on apostrophe tokens, the paper's central worked example of monosemanticity — though it does not activate on all apostrophes, since two further features cover contractions like “[I/We/They]'ll” and “[don/won/wouldn]'t” separately. Ablating it specifically suppresses the logit for the following 's' token, giving it an intuitive, predictable causal effect in contrast to the corresponding polysemantic default-basis dimension (sparse-autoencoders, §5.1, p. 7; §5.2, p. 7).

This ablation lowers a residual-stream activation's component along a feature's direction only as far as the point where the feature stops firing, not past it — respecting the same geometry, a hyperplane offset from the origin by the ReLU's negative bias, that Appendix G identifies as distinguishing dictionary features from every baseline direction. It is the tool used to measure the apostrophe feature's causal effect on next-token logits (sparse-autoencoders, §5.2, p. 7).

A layer-5 feature whose unembedding directly predicts closing-parenthesis tokens — an identity confirmable almost without cost, since layer 5 is the model's final layer, whose job is to write directions the unembedding reads out as tokens. That near-unambiguous meaning is what makes it a credible test case for tracing which upstream features (dates, acronyms, and other parenthesis-preceding phrases) cause it to fire (sparse-autoencoders, §5.3, p. 8).

A method for tracing causal dependencies between features in adjacent layers: sample contexts where a target feature strongly activates, ablate each previous-layer feature in turn, and rank features by how much their ablation lowers the target's activation, optionally recursing to build a multi-layer causal tree. Demonstrated on the closing-parenthesis feature, it is the paper's only worked example of what the future-work goal of tracing “sparse causal dependencies” actually looks like (sparse-autoencoders, §5.3, p. 7-8).

Before settling on ablation, the authors tried multiplying a layer-4 feature through the MLP's weights and checking cosine similarity against layer-5 features, hoping matrix multiplication alone would reveal causal connections. It found none, likely because the resulting vector went out-of-distribution — a negative result reported alongside the ablation-based method that did succeed, and one the authors note does not obviously generalize to the attention sublayer either (sparse-autoencoders, §D.4, p. 17).

Treating individual neurons, or individual residual-stream coordinates, as the unit of analysis is the naive baseline the paper measures its dictionary against. The apostrophe case study makes the comparison concrete: the best-matching coordinate found (dimension 21) took checking ten candidates, and even it “only explains a very small fraction of the variance for middle-to-lower activation ranges” — polysemantic where the matched dictionary feature stays clean across its whole range (sparse-autoencoders, §3.2, p. 3; §D.1, p. 16).

The transformer's running sum of layer outputs is the paper's primary object of study: sparse autoencoders are trained mainly on residual-stream activations, and the paper separately investigates whether the stream has a “privileged basis” and how outlier dimensions relate to interpretable dictionary features (sparse-autoencoders, §2, p. 2; §D.1, p. 16).

A privileged basis is a coordinate system distinguished as unusually meaningful, “such as the standard basis after a non-linearity in an MLP.” The paper states the residual stream “can usually be treated as a vector space with no privileged basis” — it is a running sum of additive outputs, not the product of an elementwise nonlinearity — which is exactly what licenses treating individual coordinates as no more principled than a random direction (sparse-autoencoders, §C.3, p. 15).

Specific residual-stream coordinates that transformers write disproportionately large, information-bearing values to, hypothesized to arise from the Adam optimizer storing gradients with finite precision in the residual basis (Dettmers et al., 2022). Searching for the apostrophe feature's best-matching default-basis dimensions, the paper finds most of the top candidates were outlier dimensions — a partial, localized exception to the residual stream's usual lack of a privileged basis, though one that still fails to make those dimensions more interpretable than a random direction (sparse-autoencoders, §D.3, p. 16-17).

An extension of the method to MLP sublayer activations rather than the residual stream, finding “mixed success”: many features are more interpretable than individual neurons, but the approach suffers from large numbers of dead features and does not yet robustly learn an overcomplete basis in intermediate or later MLP layers. The fix — a different sparsity coefficient and untied encoder/decoder weights — is itself evidence the recipe validated on the residual stream has hidden, activation-specific dependencies (sparse-autoencoders, §C.3, p. 15).

Substituting a layer's activations with the SAE's reconstruction and measuring the resulting rise in the model's perplexity on the Pile translates an abstract squared-error number into a concrete, downstream cost. For Pythia-70M's layer-2 residual stream, doing so raises perplexity from 25 to 40 — nearly doubling it, and the paper's clearest evidence that reconstruction loss, though small-looking in absolute terms, is “not yet near zero” in any sense that matters to the model's actual behavior (sparse-autoencoders, §6.2, p. 8-9).

The object all three papers keep returning to: a running sum of layer outputs with, this paper argues, no privileged basis of its own except for a handful of outlier dimensions. CAA reads it for behavioral clustering, Persona Vectors reads a single trait direction inside it, and this paper reads its basis directly — three different questions asked of the same accumulating vector space.

Elhage et al.'s (2022) aspirational goal of a complete, human-understandable enumeration of a model's features, sufficient to guarantee it will not perform dangerous behaviors such as deception. Invoked in this paper's conclusion as the long-term motivation its method is framed as a step toward — an ambition that reframes the oversight problem the rest of the corpus tackles by judging outputs: audit a model instead by listing and inspecting its internal parts.

The field named as this paper's home discipline, and the field whose “ambitious dream” enumerative safety is said to be. Where the corpus's other papers pursue model safety through training signal — reward models, constitutions, revised responses — mechanistic interpretability, as this paper frames it, pursues the same goal by reverse-engineering a network's internals well enough to check for a specific worry, deception, directly rather than by shaping outputs from the outside.

GPT-4 does the explaining in this paper, three months before CAA has it doing the rating and, in CAA, the authoring of two entire training datasets. Across both roles the corpus asks the same model to stand in for a human reader at scale — reading a feature's activating text here, reading a full generated response there — with each paper hedging the same worry that its explanations or scores may not transfer cleanly.

The metric that gives this paper its headline comparison also gives the corpus its earliest LLM-judge instance: a two-model, explain-then-simulate pipeline, adapted from Bills et al. (2023), that predates CAA's own GPT-4 rater by three months and produces the same basic shape — one model generating a description or a response, a second model checking it against ground truth or a rubric — that recurs across the corpus's next two years of papers.

The method itself, read from two years later: a way of finding directions in activation space with no target concept named in advance, in contrast to every steering-vector method the rest of the corpus builds. Persona Vectors names this contrast explicitly in its own limitations section, and puts the method to direct use, training its own sparse autoencoders on a chat model's residual stream to decompose the persona vectors it had already extracted by other means.

What Persona Vectors' sparse-autoencoder exploration is actually looking for underneath a persona vector: not the vector itself, but the smaller, more interpretable dictionary features a much larger, unsupervised dictionary decomposes it into. The same unit of analysis this paper defines and validates on Pythia in 2023 is, two years later, being read out from underneath a very different kind of direction on a much larger model.

Principal Component Analysis appears throughout the corpus as general-purpose, off-the-shelf machinery repurposed for whatever a paper needs. This paper measures it as a baseline decomposition, finding sparse dictionary features beat it on both autointerpretability score and IOI causal-localization precision. CAA instead uses it as a visualization tool, projecting activations to two dimensions to check by eye whether a behavior separates — the same technique, doing diagnostic work at two different grains of the same underlying question.

The autoencoder takes a language model's cached activation vector x and produces hidden coefficients c = ReLU(Mx + b), then reconstructs x-hat = M^T c using the same matrix M, tied between encoder and decoder. M's rows are read off afterward as the dictionary features under study. Training minimizes reconstruction error plus an L1 sparsity penalty on c (Equation 4). The method is applied mainly to residual-stream activations from the Pythia (model suite) suite: Pythia-70M (dimension 512) and Pythia-410M (dimension 1024). Because training happens after the fact on frozen, cached activations, the approach stays unsupervised and never touches the underlying language model's own weights or training.

Each row of the trained matrix M is a single dictionary feature: a direction in activation space that contributes to the reconstruction whenever its coefficient is nonzero. Because the sparsity penalty forces M's rows to unit norm, a feature's row is fixed as a direction, and its activation strength alone determines how much it contributes on a given input. Every later result in the paper, interpretability scores, causal patching, and the case studies, is a claim about properties of these individual rows, not about the autoencoder as a whole.

The training objective (Equation 4) adds alpha times the L1 norm of c to the squared reconstruction error, penalizing how many features fire at once rather than the size of the reconstruction error directly. Alpha is the paper's central knob: raising it produces sparser, more feature-like activations at the cost of reconstruction accuracy, and its value is set separately for the residual stream (8.6e-4) and the MLP sublayer (3.2e-4), based on the active-feature sweeps in the next chapter.

Tying the decoder to the encoder's transpose halves the parameter count and encodes the assumption that the direction which detects a feature and the direction that defines its effect on the reconstruction should coincide. The paper reports tied weights do not hurt performance when training on residual-stream activations, but do reduce performance on MLP activations, motivating an untied variant with separate encoder and decoder matrices used specifically for MLP-sublayer training.

R sets the autoencoder's hidden size as dhid = R times din, controlling how overcomplete the learned dictionary is relative to the model's own activation dimension. The paper sweeps R from 0.5 (undercomplete) to 32 (far overcomplete) and finds average per-feature interpretability depends far more on layer than on R, the smallest dictionary tested scores comparably to ones dozens of times larger, while a separate quantity, the share of a layer's activation variance the dictionary explains overall, does keep rising with R. A bigger dictionary explains more of the layer without any single feature in it becoming more interpretable.

Sweeping alpha produces a smooth curve trading the number of simultaneously active features against unexplained reconstruction variance, with no distinct bend separating a correct operating point from an incorrect one. The paper reads this absence of a knee as weak evidence against there being one single, objectively correct sparse decomposition of a given layer's activations, a caveat that applies to every dictionary used elsewhere in the paper, since none of them is picked out as uniquely privileged by the data.

A dictionary feature that never activates across the training corpus is dead, wasted capacity that never gets read as a feature. The paper finds residual-stream dictionaries stay almost fully alive up to roughly a 4x expansion factor before features start dropping out, which is what let it pick the alpha values used throughout, 8.6e-4 for the residual stream and 3.2e-4 for the MLP. MLP-sublayer dictionaries fare much worse: because the MLP's nonlinearity leaves large regions of input space unused, many features get stuck barely ever firing even with no sparsity penalty at all, in some cases leaving less than half a dictionary usably alive.

The squared-error term in Equation 4 is never driven to zero in any of the paper's trained autoencoders, meaning the dictionary always fails to capture some information present in the original activation. The paper quantifies this gap independently with a perplexity check: substituting Pythia-70M's layer-2 residual stream with its autoencoder reconstruction raises the model's perplexity on the Pile from 25 to 40, confirming that reconstruction loss reflects a real, measurable loss of information rather than a training artifact.

The score (Bills et al., 2023) is produced in three steps: sample text fragments from the first 50,000 lines of OpenWebText where a feature activates, have GPT-4 write a natural-language explanation from a handful of those fragments and their per-token activations, then have GPT-3.5 use that explanation to predict the feature's activation on held-out fragments, GPT-3.5 stands in for GPT-4 as the simulator only because its public API exposed the log-probabilities the simulation step needs. The correlation between predicted and true activations is the feature's score, so a score can run negative when the written explanation performs worse than no explanation at all.

PCA projects the same activation dataset onto directions of maximal variance, computed with an online estimator over the same quantity of data used to train the autoencoders. It is one of four baselines run against sparse coding in the headline comparison, alongside Independent Component Analysis (ICA), the default neuron basis, and random directions, and it is the baseline the paper carries forward into the causal-patching chapter below. PCA trails sparse coding on both measures: it scores lower on autointerpretability at every layer, and later requires more patched directions and larger edit magnitudes to reach the same causal effect on the indirect-object-identification task.

The default scoring mixes high-activating fragments with random ones, top-and-random scoring, which could make scoring artificially easy. Rerunning with random-only scoring, fragments drawn with no regard to where a feature activates, drives every method's absolute score down, since a small random sample of a sparse feature's activations often contains no strongly-activating example at all. Sparse coding still sits clearly above all four baselines under this harsher test, especially in the earlier layers, ruling out an artifact-of-easy-scoring explanation for the headline gap.

Applying the same method to MLP sublayer activations instead of the residual stream gives mixed success: many learned features are still more interpretable than individual neurons, but the large number of dead features MLP dictionaries accumulate means the living dictionary often fails to be genuinely overcomplete, and the training pipeline does not yet robustly learn overcomplete bases in the MLP's middle and later layers. Under the stricter random-only scoring, sparse coding's MLP advantage becomes inconsistent, at layer 3 it drops to about 0.03, below the Identity ReLU neuron baseline, where the residual-stream results hold up cleanly across layers.

Correlating each feature's interpretability score against statistical moments of its activation distribution, pooled across all layers and five dictionary sizes, finds skew and kurtosis are the only two moments with a positive, non-negligible correlation, 0.20 and 0.15 in the table (the body text separately quotes 0.19 and 0.24), while mean and variance are essentially uncorrelated. Because ICA explicitly searches for maximally non-Gaussian, heavy-tailed directions, a search that can itself be reformulated as maximizing the L4 norm, this correlation is offered as the explanation for why ICA is consistently the strongest of the non-dictionary-learning baselines throughout the paper's comparisons.

A sparse dictionary feature activates on only a fraction of inputs, while a raw PCA or ICA direction is active across an entire half-space, so the interpretability gap could in principle just reflect these baselines being compared unfairly against a sparser method. The control restricts PCA and ICA to a fixed number K of simultaneously active directions per datapoint, matched to sparse coding's own average active-feature count. This adjustment does not close the gap: the Top-k variants track close to their unrestricted counterparts rather than rising toward sparse coding's score, so unequal activation density explains only a small part of the headline result.

Activation patching (Vig et al., 2020) edits a model's internal activations toward values observed on a counterfactual input and measures the resulting change in output, a way to test whether a proposed direction actually carries the causal information behind a behavior, not just a correlate of it. The paper adapts it to the Indirect Object Identification task (Wang et al., 2022), in which a model completes sentences like 'Then, Alice and Bob went to the store. Alice gave a snack to ___', chosen because the underlying mechanism was already characterized by prior work, letting the comparison isolate how precisely each decomposition's directions localize it rather than whether the mechanism exists at all.

The procedure, an application of activation patching to individual dictionary features, runs the model once on a counterfactual sentence to cache each feature's encoded activation, then re-runs it on the base sentence, adding the cached activation differences for a chosen feature subset into the residual stream at one layer, and measures the KL divergence between the patched output and the true counterfactual target. On layer 11 of Pythia-410M, sparse-dictionary features reach a given KL divergence with fewer patched directions and a smaller total edit magnitude than PCA components, and that advantage disappears for a non-sparse (alpha=0) dictionary, evidence that sparsity itself, not just being a learned dictionary, drives the improved localization.

Rather than patching an arbitrary or fixed set of directions, the paper orders candidate features by individually-estimated causal importance using the ordering procedure from Automated Circuit Discovery (Conmy et al., 2023, Algorithm 4.1), treating each feature as an independent contributor to the KL-divergence metric averaged over 50 IOI test examples. The resulting ranking, computed separately for each decomposition, dictionary features and PCA alike, is what determines the order in which directions are added along the x-axis of the patching curves, so the comparison reflects each method's most causally important directions first, not a random subset.

Dictionary feature 556 activates almost exclusively on the apostrophe token, and using less-than-rank-one ablation, lowering the residual stream's component along the feature's direction only until the feature stops firing, to remove it from the residual stream mainly suppresses the model's predicted logit for the following 's' token, matching how apostrophes precede 's' in contractions and possessives like 'let's'. The feature does not fire on every apostrophe: two further dictionary features fire on apostrophes only in specific contraction families, one for pronoun contractions like 'I'll' and 'you'll' (ablation suppresses a broader family of endings), and one for negation contractions like 'don't' and 'won't' (ablation narrowly suppresses mainly the 't' that closes '-n't'). That the dictionary represents a single character with several distinct, narrowly-scoped features is itself evidence for fine-grained Monosemanticity.

The residual stream's own coordinate basis has no particular reason to align with meaningful features, and the paper shows this concretely: the default-basis dimension that most resembles the apostrophe feature, the 10th-highest dimension the apostrophe feature itself reads from, looks like an apostrophe detector only in its highest activation bins. Across the fuller range it fires on many unrelated tokens ('and', 'one', 'part', 'number'), the textbook picture of polysemanticity, and the dimensions it draws on turn out to be outlier dimensions rather than typical basis coordinates. The clean, single-token behavior of dictionary feature 556 is therefore a property of the learned feature, not of any one default-basis direction.

The property the case studies are built to demonstrate is monosemanticity: a feature activating for one coherent, human-interpretable pattern rather than several unrelated ones, the converse of the Polysemanticity the paper's method targets. Two further worked examples show this pattern recurs outside apostrophes: a feature that fires on the word 'if' specifically in programming contexts, whose ablation suppresses code-syntax tokens that typically follow it ('(', '&&', 'then'), and a feature that fires on the 'Dis' prefix within specific bigrams, whose ablation suppresses the word-completions that follow it ('CLAIM', 'closed', 'ordered'). In both cases the input tokens that trigger the feature and the output tokens its removal suppresses tell a matching causal story.

Beyond a single feature's inputs and outputs, the paper traces causal dependencies across layers: given a target feature, sample contexts where it strongly activates, then for each feature in the previous layer, ablate it and measure how much the target's activation drops, ranking previous-layer features by that effect and optionally recursing. Applied to the layer-5 closing-parenthesis feature, whose unembedding directly predicts closing-parenthesis tokens, this produces a multi-layer causal tree with two branches feeding the root, one tracking generic parenthetical context, one tracking acronyms specifically, that merge just before the final feature. The paper contrasts this success with a weight-based alternative tried between layers 4 and 5, multiplying a feature through the MLP weights and checking cosine similarity, which found no meaningful connections, likely because the resulting vector left the model's typical activation distribution.