Autointerpretability score
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
An automated interpretability metric (Bills et al., 2023) in which a language model is shown text where a feature activates, writes a natural-language explanation of the feature, and then a (possibly different) language model uses that explanation to predict the feature's activations on held-out text; the correlation between predicted and actual activations is the score. This paper applies it at scale (GPT-4 generates explanations, GPT-3.5 simulates activations) as its primary measure of dictionary-feature interpretability, run across thousands of features and several baselines.
A sparse autoencoder can turn out thousands of learned features in a single layer, far more than any team could read through by hand, so a paper claiming its features are interpretable needs a way to check that claim at scale rather than by spot-checking a handful and hoping the rest look similar. The autointerpretability score is that stand-in: one model writes a plain-language explanation of what a feature detects, a second model tries to predict the feature's activations from that explanation alone, and the correlation between prediction and reality becomes the number.
This page walks through the pipeline behind that number, the sampling choices that shape what it can measure, and the claims it is used to defend alongside blind spots — like causal effects on a model's output — it was never built to catch.
A scalable stand-in for reading every feature by hand
The paper's central problem is that its sparse autoencoders learn thousands of dictionary features per layer, far too many for a human to read through and judge one by one. The autointerpretability score, adopted from Bills et al. (2023), is the metric that makes evaluating all of them possible: a scalable, automated proxy for whether a feature shows "reduced polysemanticity," i.e. Monosemanticity (sparse-autoencoders, §"3.1 INTERPRETABILITY AT SCALE", p. 3). It becomes this paper's primary yardstick throughout Section 3, used to argue that sparse dictionary features beat Principal Component Analysis (PCA), Independent Component Analysis (ICA), the Default (neuron/residual-stream) basis baseline, and the Random directions baseline, and it is stress-tested repeatedly in the appendices against the worry that the comparison is somehow rigged in the dictionary's favor.
Two models, one explanation, one blind prediction
The score comes from a five-step pipeline (sparse-autoencoders, §"A AUTOINTERPRETATION PROTOCOL", p. 11): a feature's activation is measured across 50,000 OpenWebText fragments, rescaled to integers 0-10, and the 20 most-activating fragments are found. GPT-4 is shown only 5 of these fragments together with their per-token activations and writes a single free-text explanation of what the feature detects. GPT-3.5 then receives nothing but that explanation -- no fragments, no numbers -- and must predict per-token activations on a fresh batch of 5 highly-activating and 5 randomly sampled fragments, 640 tokens in total (sparse-autoencoders, §"A AUTOINTERPRETATION PROTOCOL", p. 12). The score is the correlation between GPT-3.5's predicted activations $\hat{a}$ and the feature's true activations $a$ across those tokens: $$\text{score} = \mathrm{corr}(a, \hat a) = \frac{\mathrm{cov}(a,\hat a)}{\sigma_a\,\sigma_{\hat a}}.$$ GPT-3.5 fills this simulator role only because, at the time, OpenAI's public API returned per-token logprobs for GPT-3.5 but not for GPT-4, which Bills et al. (2023)'s original protocol had used for simulation too (sparse-autoencoders, §"A AUTOINTERPRETATION PROTOCOL", p. 11).
What a high or low score is, and isn't, evidence of
Sampling matters as much as scoring: because most features fire on only a sliver of the corpus, the default mixes top-activating fragments with random ones (Top-and-random vs. random-only interpretability scoring) rather than sampling purely at random, and Appendix C.4's finding that scores correlate with a feature's skew and kurtosis (Interpretability-vs-kurtosis/skew correlation analysis) is offered as the reason heavy-tailed features are favored. The score also has known blind spots: it never looks at how a feature causally affects the model's output, so late-layer features -- often best explained by their effect on the logits rather than by input text -- score worse for reasons that may have little to do with quality (sparse-autoencoders, §"3.2 SPARSE DICTIONARY FEATURES ARE MORE INTERPRETABLE THAN BASELINES", p. 5). It is also a different axis of fidelity entirely from the Reconstruction-substitution perplexity metric, which asks how much information a dictionary loses when it replaces a layer's activations outright rather than whether an individual feature is describable in words.