Steering outside the residual stream (proposed extension)
Steering Llama 2 via Contrastive Activation Addition — introduced
A suggested future direction proposing to apply CAA-style interventions at other points in the model, such as after the MLP but before it merges into the residual stream, to help localize where behavioral representations live.
CAA adds its steering vector to the residual stream, the running total every layer reads from and writes back into, but that's not the only place inside a transformer where a behavior's representation might live. Intervening somewhere more specific, after a sublayer's own computation but before it merges back into that running total, could reveal something the residual stream alone can't.
That proposed extension is set out below: applying CAA-style perturbations at other points in the model, such as right after the MLP, as a way to localize where behavioral representations actually sit rather than just where they can be nudged.
Trading convenience for localization
This is another proposed extension confined to the Discussion, also untested in the paper. The Residual stream is a running sum: every attention block and MLP writes its output into the same accumulating vector, so an intervention added at layer $L$ perturbs the combined total of everything written into the stream so far, without distinguishing which sub-component's contribution is being altered. The proposal is to intervene instead "at other points in the model, such as after the MLP but before merging into the residual stream" (contrastive-activation-addition, §"9.1 Suggested future work", p. 9) — a site that acts on one specific computation's output rather than an accumulated total.
A diagnostic payoff, not just a practical one
The stated motivation is explicitly interpretive as well as practical: by moving the intervention to different pre-merge points, "we could learn more about where representations are localized in the model and achieve more targeted effects" (contrastive-activation-addition, §"9.1 Suggested future work", p. 9). The residual stream's summing structure is exactly what makes Contrastive Activation Addition (CAA)'s current intervention easy to apply — one vector, one layer, added into a stream every block already writes into — and hard to localize; this proposal, alongside Steering at targeted token positions (proposed extension), names the two axes, where in the network and at which tokens, the paper thinks its own method would need to move along to become more precise.
A site already probed elsewhere, for a different purpose
Intervening outside a single-layer residual stream is not unprecedented as a site — In-Context Vectors (ICV) already steers attention activations rather than the residual stream, and does so at every transformer layer simultaneously rather than one. That precedent shows the general move is workable, but it is not the same proposal: ICV trades away CAA's single-layer specificity entirely in exchange for broad coverage, while this suggestion keeps the single-layer, single-site precision that makes CAA's later interpretability analyses possible, and only relocates that one site from the summed residual stream to an individual sub-component's output, such as the MLP, before it is folded in.