Superposition — motivates → Sparse autoencoders (SAEs)

The superposition hypothesis reframes 'find interpretable units' as a concrete recovery problem: if activations are sparse linear combinations of unknown ground-truth features, then finding those features is the classic sparse dictionary learning problem of Olshausen & Field (1997), and both empirical work (Sharkey et al., 2023) and theory (Wright & Ma, 2022) show that l1-penalized reconstruction can recover the true generating features. That equivalence, not just the general goal of 'fixing superposition,' is what licenses the paper's specific architectural choices: a hidden layer of size dhid = R*din so the dictionary can be overcomplete the way superposition requires, and an l1 penalty on the hidden coefficients rather than on the weights. In other words, superposition does not just motivate building some interpretability tool -- it dictates the shape of this one.