Residual stream
Steering Llama 2 via Contrastive Activation Addition — inherited; Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
The running sum of layer outputs that a transformer's activations flow through and accumulate in. CAA computes and injects its steering vectors directly into this stream at a chosen layer.
Every layer of a transformer reads from and writes back into the same running total of information as it flows through the network, a channel called the residual stream. It's an unglamorous piece of architecture, but it's also the one place a steering vector can be added once and have its effect persist through every layer that comes after.
What the residual stream is, and why CAA targets it directly, gets explained ahead: injecting a steering vector into this stream at a single chosen layer rather than intervening on any more specific internal component.
The running total every layer writes into
In a transformer, the residual stream is the vector that carries information from one layer to the next by simple addition: each attention block and each MLP block reads the current stream, computes some update, and adds that update back in, rather than replacing the stream outright. A token's representation at layer $L$ is literally a running sum of every block's output from layer 1 through $L$, plus the original embedding — nothing already written is overwritten, only added to. That additive structure is what makes the stream a natural place to intervene: adding an external vector to it is the same operation every layer already performs. It is also why the stream's norm tends to grow deeper into the network, accumulating rather than resetting, a pattern the paper cites directly (Heimersheim and Turner, 2023) (contrastive-activation-addition, §"Vector normalization choices", p. 10).
Where CAA writes, and why that has consequences
Contrastive Activation Addition (CAA) commits to injecting its steering vector into the residual stream at one chosen layer, rather than into attention activations at every layer, as In-Context Vectors (ICV) does, or a sparse set of attention heads, as Inference-Time Intervention (ITI) does. Because the stream's norm grows with depth, and Steering vector normalization choices standardizes vector magnitude across behaviors but deliberately not across layers, a fixed Steering multiplier represents a different relative perturbation depending on which layer it is added at (contrastive-activation-addition, §"Vector normalization choices", p. 10) — the residual stream is not a neutral injection site, its own scale shapes how the paper's layer-optimality results should be read.
Uninformative, then abruptly not
The stream is not uniformly informative about behavior across depth. PCA visualization of contrastive activations shows Behavioral clustering is largely absent for roughly the first third of the model's layers, then appears abruptly within a single layer: at layer 9 of Llama 2 7B Chat there is no separation on the refusal dataset, while at layer 10 there is a clean split (contrastive-activation-addition, §"3.2 Visualizing activations for contrastive dataset analysis", p. 4). Steering outside the residual stream (proposed extension) is the paper's own proposed alternative to injecting into this accumulating sum — intervening at a sub-component's output, such as after the MLP but before it merges into the stream, to localize effects the summed stream currently blurs together.