Weight-based feature connection attempt (failed method)
Sparse Autoencoders Find Highly Interpretable Features in Language Models — introduced
A failed alternative method for connecting features across layers, tried by multiplying a layer-4 feature through the MLP weights and checking cosine similarity with layer-5 features. It found no meaningful connections, likely because the resulting vector went out-of-distribution; reported as a negative result contrasting with the ablation-based feature-circuit-detection method that did succeed.
Not every method in an interpretability paper works, and this one is reported precisely because it didn't: an attempt to find which downstream feature a given feature feeds into using nothing but the network's weights, no forward passes or real activations required, promising if it had worked because it would have been almost free to compute.
The page covers what was actually tried — multiplying a feature through a layer's weights and checking similarity against the next layer's dictionary — why the resulting vector turned out to be meaningless because it never landed back in the space the network actually operates in, and what the more expensive, ablation-based method that replaced it does differently.
A documented negative result inside the causal-proof theme
Weight-based connection attempt is one of the few places in the paper where a method is reported precisely because it failed. It belongs to the causal-proof-by-patching theme as the road not taken: an attempt to establish causal links between features across layers algebraically, from weights alone, rather than empirically, by ablating and observing a real forward pass.
What was tried: multiply through the MLP, check cosine similarity
The authors attempted a weight-based method for connecting the layer-4 dictionary to the layer-5 dictionary: take a layer-4 feature, multiply it through the MLP's weight matrix, and check the cosine similarity of the resulting vector against every feature in the layer-5 dictionary, hoping that a strong similarity would reveal which downstream feature the layer-4 feature feeds into (sparse-autoencoders, §"D.4 Failed interpretability methods", p. 17). The appeal of the approach was its cost: no forward passes, no sampling of activating contexts, no ablations -- just one matrix multiplication and a similarity check per feature pair.
Why it didn't work
The method turned up no meaningful connections at all (sparse-autoencoders, §"D.4 Failed interpretability methods", p. 17), and the paper's own explanation is that the intermediate vector produced by multiplying a feature through the MLP's weights goes out of distribution: it does not land back in the region of activation space the MLP was ever trained to process, so a cosine similarity computed there is not measuring anything the network's actual computation respects. The authors also flag a structural limitation: the approach doesn't obviously generalize to the attention sublayer, since there is no single position dimension a feature vector can be cleanly multiplied through the way there is for an MLP (sparse-autoencoders, §"D.4 Failed interpretability methods", p. 17).
What replaced it
Feature-circuit detection, the method actually used to trace the closing-parenthesis feature's upstream causes, trades away the weight method's one-shot cheapness for causal grounding: instead of asking what a feature's weights point toward algebraically, it asks what happens to a target feature's real activation when a candidate cause is ablated from an actual forward pass. Reporting this failure alongside the method that succeeded is itself informative -- it ties to the paper's broader Appendix G finding that dictionary features' apparent structure is bound tightly to the activation space the model actually inhabits, not to weight-space algebra performed on top of it.