Polysemanticity
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
The phenomenon in which individual neurons (or other analysis units) activate in multiple, semantically unrelated contexts, preventing concise human-understandable explanations of a network's internal computation. Named as the paper's central roadblock in the abstract; the paper's entire method is aimed at finding a decomposition of activation space with reduced polysemanticity.
A single neuron lighting up for several completely unrelated ideas sounds like a minor curiosity until it becomes the reason nobody can write down, in plain language, what that neuron is actually for. This mismatch between one unit and many meanings is named as the very first obstacle standing between researchers and a real understanding of what a network computes.
The rest of this entry traces where the term comes from, why relying on individual neurons as the basic unit of analysis breaks down once the mismatch is taken seriously, and what a polysemantic unit looks like mechanically: a single readout responding to several unrelated underlying signals at once.
The roadblock the paper opens with
Polysemanticity is the phenomenon in which a single neuron, or any other individual unit of analysis inside a neural network, activates across several unrelated concepts rather than tracking one coherent idea. The term comes from the Anthropic interpretability line of work: Olah et al. (2020) first documented individual units firing for conceptually unrelated stimuli during manual circuit analysis, and Elhage et al. (2022b) later proposed superposition as its structural explanation. This paper names polysemanticity as its chosen problem statement from the first line of its abstract: 'One of the roadblocks to a better understanding of neural networks' internals is polysemanticity' (sparse-autoencoders, §"ABSTRACT", p. 1). Framed this way, polysemanticity is not a side observation but the obstacle the entire sparse-autoencoder method is built to remove.
Why a broken unit of analysis blocks the whole research program
Mechanistic interpretability proceeds by decomposing a network into small units that can be studied one at a time; the paper notes that using individual neurons as those units 'has had some success' (Olah et al., 2020; Bills et al., 2023) (sparse-autoencoders, §"1 INTRODUCTION", p. 1), which is why neurons were the default unit before this paper's method existed. Polysemanticity undercuts exactly that method: a neuron firing for several unrelated concepts cannot be assigned a single human-readable role no matter how carefully it is probed in isolation. Superposition is the paper's inherited hypothesis for why polysemanticity arises at all, and resolving it, by finding a better unit of analysis than the neuron, is the explicit goal behind training Sparse autoencoders (SAEs).
What a polysemantic unit looks like mechanically
Concretely, a neuron's activation is a linear readout of the layer's representation, $z = w \cdot x$ for some weight vector $w$ and activation vector $x$. If $x$ actually encodes several features additively, $x = \sum_i a_i g_i$, where each $a_i$ indicates the presence of feature $g_i$, then $z = \sum_i a_i (w \cdot g_i)$. Whenever $w$ has a substantial dot product with more than one $g_i$, the neuron's activation rises whenever any of those features is present, regardless of whether the underlying concepts have anything in common. This is exactly what the paper's case studies document at the level of individual Residual stream coordinates: a single scalar that lights up for several unrelated tokens or contexts (sparse-autoencoders, §"5.1 INPUT: DICTIONARY FEATURES ARE HIGHLY MONOSEMANTIC", p. 7), the mirror image of the Monosemanticity the paper's dictionary features are shown to approximate more closely.