Sparse autoencoders (SAEs) — runs counter to the depth profile of → Residual stream

Both papers plot something against depth in the same architectural object, and the two curves point opposite directions. The sparse-autoencoder paper’s interpretability advantage over PCA, ICA, and the default basis is measured layer by layer and is not constant: it is already "comparable to ICA" by layer 4 and shows "minimal improvement in the final layer" (sparse-autoencoders, §"3.2 Sparse dictionary features are more interpretable than baselines", p. 5) — and its dictionaries are trained with no knowledge of any downstream task, since "the training of our feature dictionaries does not emphasize any particular task" (sparse-autoencoders, §"4 Identifying causally-important dictionary features for indirect object identification", p. 6). Contrastive Activation Addition, reading the same residual stream for a different purpose three months later, finds that behavior-relevant structure appears late rather than early: PCA-projected activations show no separation by behavior at layer 9 of Llama 2 7B Chat and a clean split one layer later, with behavioral clustering largely absent for roughly the first third of the network’s depth (contrastive-activation-addition, §"3.2 Visualizing activations for contrastive dataset analysis", p. 4). The two findings are not strictly in tension — one is a decomposition’s fidelity to arbitrary local structure, the other is where one specific labeled behavior becomes linearly separable — but juxtaposed they mean a dictionary’s best guarantee of monosemanticity and a behavior’s best guarantee of linear separability sit at opposite ends of the same stack of layers, a fact visible only once a later paper supplied the second curve.