Outlier dimensions
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
Specific residual-stream coordinates that transformers tend to write disproportionately large, information-bearing values to, hypothesized (Dettmers et al., 2022) to arise from the Adam optimizer storing gradients with finite precision in the residual basis. The paper finds that its top-activating default-basis dimensions for both the most positive and most negative apostrophe-related directions were outlier dimensions rather than typical basis coordinates.
A handful of coordinates in a model's internal representation can carry values wildly larger than all the others, disproportionate enough to complicate compressing the model down to lower precision — and it's tempting to assume anything that stands out numerically this much must also be unusually meaningful.
From here the entry traces where these outsized coordinates come from, how a later paper independently rediscovers them while hunting for the dimensions behind one of its own case studies, and why carrying an outsized share of a layer's magnitude turns out not to make a coordinate any easier for a person to interpret.
A documented exception to 'no privileged basis'
Outlier dimensions are the handful of coordinates in a transformer's activations that reliably carry disproportionately large-magnitude values relative to typical coordinates. The phenomenon was characterized at scale by Dettmers et al. (2022), who found it complicated low-precision quantization of large language models because a few outlier features dominate a layer's dynamic range; the proposed mechanism, cited by this paper, is that the Adam optimiser stores gradients with finite precision specifically in the residual-stream basis (Elhage et al., 2023). This paper inherits the concept to interpret its own Default (neuron/residual-stream) basis baseline: when it searches the residual stream's raw coordinates for the dimensions most associated with its apostrophe example, the results are dominated by outlier dimensions rather than typical ones (sparse-autoencoders, §"D.3 FEATURE SEARCH DETAILS", p. 17).
What they explain, and what they don't
Outlier dimensions sit in tension with the paper's broader finding that the Residual stream mostly lacks a Privileged basis: because Adam's optimisation dynamics single out these specific coordinates as unusually information-bearing, they are a partial, localized counterexample rather than a wholesale one. But carrying real information is not the same as being individually legible: despite the outlier dimensions' outsized share of the residual stream's dynamic range, the paper directly tests whether default-basis directions score better than chance on interpretability and reports the opposite, 'we do not find residual stream basis directions to be any more interpretable than random directions' (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15). The information these coordinates carry appears to remain spread thinly across many overlapping instances of Polysemanticity rather than concentrated into a form a human explainer can read off directly.
How the paper found them
The paper located its outlier dimensions by direct search rather than by reusing Dettmers et al.'s original quantization diagnostic: given a sentence ending in an apostrophe, it looked for which residual-stream coordinates activate most strongly, in both directions, on that final token, then checked how many of the extreme coordinates matched already-known outlier dimensions. It found that 'the top two most positive dimensions were outlier dimensions, the top two negative dimensions were our displayed one and another outlier dimension, respectively' (sparse-autoencoders, §"D.3 FEATURE SEARCH DETAILS", p. 17). Three of the four most extreme coordinates in either direction turning out to be previously-identified outlier dimensions is the paper's own direct evidence that these coordinates are non-generic, even though it does not by itself make them interpretable.