SAE training on the MLP sublayer
Sparse Autoencoders Find Highly Interpretable Features in Language Models — introduced
An extension of the paper's method to train dictionaries on MLP sublayer activations rather than the residual stream, finding 'mixed success': many features are more interpretable than individual neurons, but the approach suffers from large numbers of dead features and does not yet robustly learn overcomplete bases in intermediate or later MLP layers.
A method validated on one part of a model doesn't automatically work on another, especially when that part comes with a complication the first didn't have: a coordinate system that might already be meaningful before any dictionary is trained on it. Pointing the same recipe at different activations was the paper's way of testing that assumption rather than assuming it.
This entry traces what changed once the target became a space competing against neurons with a genuine structural claim to interpretability, why the result reads as mixed success rather than a clean win, and which fixes the paper adopted, and left unfinished, against a far worse version of a familiar failure there.
Testing whether the recipe generalizes beyond the residual stream
Everything demonstrated for the paper's method in the main text is about the residual stream. The paper is explicit that its approach 'can, in principle, be applied to any set of internal activations of a language model, not just the residual stream' (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15), and MLP-sublayer training is the appendix experiment that actually tests that claim, rather than assuming it. This matters because the residual stream is the one part of the model the paper treats as having no Privileged basis to compete against; the MLP hidden layer, sitting right after a nonlinearity, is the paper's own example of a space that does have one, so training a dictionary there means competing against neurons that already have some claim to being meaningfully aligned with the coordinate axes (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15).
What the stress test found
The result is 'mixed success': many MLP dictionary features are still more interpretable than the neurons themselves, evidence that the privileged basis does not make individual neurons optimal even where they have some structural advantage (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15). But two of the working recipe's assumptions break at once. Large numbers of features go dead -- the Dead features problem is far worse here than on the residual stream, severe enough that a dictionary with twice as many features as neurons can end up with fewer than half of them usably alive, except in early layers. And the training pipeline does not yet reliably learn a genuinely overcomplete basis in the middle or later MLP layers at all. The partial fix the paper adopts -- a different sparsity coefficient (alpha=3.2e-4 rather than the residual stream's 8.6e-4) and untied encoder/decoder matrices instead of the residual-stream default -- only partially restores results, and is itself evidence that the sparse-autoencoder recipe validated on the residual stream is not activation-agnostic: it has hidden dependencies on which part of the model it is applied to (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15).
What counts as the MLP-training setting, concretely
MLP SAE training uses the same core architecture and loss as the residual-stream case, but with $x$ drawn from MLP-sublayer activations rather than the residual stream, and, per the fix above, an untied forward pass $c = \mathrm{ReLU}(M_ex+b)$, $\hat{x} = M_d^Tc$ in place of the tied default. Results are reported using an L1 coefficient alpha=3.2e-4 and dictionary size ratio R=1 (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15), smaller values on both axes than the residual-stream defaults, reflecting how much harder this setting is to train well. The paper frames the whole effort as unfinished business rather than a negative result: it reports being 'currently working on methods to overcome' the dead-feature and overcompleteness problems specific to middle and later MLP layers (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15).