Residual stream
Steering Llama 2 via Contrastive Activation Addition — inherited; Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited; Sparse Autoencoders Find Highly Interpretable Features in Language Models — refined
The running sum of layer outputs that a transformer's activations flow through and accumulate in. CAA computes and injects its steering vectors directly into this stream at a chosen layer.
Every layer of a transformer reads from and writes back into the same running total of information as it flows through the network, a channel called the residual stream. Because that channel is built by simple addition rather than any operation that singles out a coordinate, it's not obvious the raw coordinates mean anything on their own — and different papers end up treating the stream in almost opposite ways.
From here the entry covers whether its coordinates carry any privileged meaning versus being just one arbitrary rotation among many, and how it gets used two different ways across this corpus: as a fixed site to inject a steering vector into, and as raw material decomposed back into a dictionary of interpretable features.
The running total every layer writes into
In a transformer, the residual stream is the vector that carries information from one layer to the next by simple addition: each attention block and each MLP block reads the current stream, computes some update, and adds that update back in, rather than replacing the stream outright. A token's representation at layer $L$ is literally a running sum of every block's output from layer 1 through $L$, plus the original embedding — nothing already written is overwritten, only added to. That additive structure is what makes the stream a natural place to intervene: adding an external vector to it is the same operation every layer already performs. It is also why the stream's norm tends to grow deeper into the network, accumulating rather than resetting, a pattern the paper cites directly (Heimersheim and Turner, 2023) (contrastive-activation-addition, §"Vector normalization choices", p. 10).
Whether this running sum has a privileged basis
Because the residual stream accumulates layer outputs by simple addition rather than through an elementwise nonlinearity, no single coordinate system is architecturally singled out the way the standard basis is after a ReLU inside an MLP: rotate the stream's coordinates and a linear sum rotates along with them, unaffected. The 2023 sparse-autoencoder paper investigating this stream states the default expectation directly: the residual stream 'can usually be treated as a vector space with no privileged basis' (sparse-autoencoders, §"C.3 INTERPRETING THE MLP SUBLAYER", p. 15), which licenses treating any individual coordinate, the paper's default-basis baseline, as no more principled a unit of analysis than a random direction. The one documented exception is a cluster of Outlier dimensions, coordinates transformers write disproportionately large values to, attributed to the Adam optimiser's finite-precision gradient storage; even so, the paper finds these coordinates no more interpretable than chance, so the exception restores information-bearing status to a few coordinates without restoring legibility.
Where CAA writes, and why that has consequences
Contrastive Activation Addition (CAA) commits to injecting its steering vector into the residual stream at one chosen layer, rather than into attention activations at every layer, as In-Context Vectors (ICV) does, or a sparse set of attention heads, as Inference-Time Intervention (ITI) does. Because the stream's norm grows with depth, and Steering vector normalization choices standardizes vector magnitude across behaviors but deliberately not across layers, a fixed Steering multiplier represents a different relative perturbation depending on which layer it is added at (contrastive-activation-addition, §"Vector normalization choices", p. 10) — the residual stream is not a neutral injection site, its own scale shapes how the paper's layer-optimality results should be read.
Uninformative, then abruptly not
The stream is not uniformly informative about behavior across depth. PCA visualization of contrastive activations shows Behavioral clustering is largely absent for roughly the first third of the model's layers, then appears abruptly within a single layer: at layer 9 of Llama 2 7B Chat there is no separation on the refusal dataset, while at layer 10 there is a clean split (contrastive-activation-addition, §"3.2 Visualizing activations for contrastive dataset analysis", p. 4). Steering outside the residual stream (proposed extension) is the paper's own proposed alternative to injecting into this accumulating sum — intervening at a sub-component's output, such as after the MLP but before it merges into the stream, to localize effects the summed stream currently blurs together.
An object to decompose, not just an object to inject into
Where Contrastive Activation Addition (CAA) and Persona vector both treat the residual stream chiefly as a site to write a direction into, the 2023 sparse-autoencoder paper (Cunningham et al.) treats it chiefly as an object to read: it trains sparse autoencoders mainly on residual-stream activations from Pythia-70M and Pythia-410M (sparse-autoencoders, §"2 TAKING FEATURES OUT OF SUPERPOSITION WITH SPARSE DICTIONARY LEARNING", p. 2), aiming to recover the overcomplete set of features Superposition hypothesises the stream packs into its comparatively few dimensions. This shift in role also surfaces an opposite depth pattern from CAA's: the sparse-autoencoder paper's interpretability advantage over baselines is largest in early layers and 'comparable to ICA' by layer 4, showing 'minimal improvement in the final layer' (sparse-autoencoders, §"3.2 SPARSE DICTIONARY FEATURES ARE MORE INTERPRETABLE THAN BASELINES", p. 5), while CAA finds behaviorally relevant structure appears late, with clustering by behavior largely absent for roughly the first third of the network's depth before appearing abruptly around layer 9-10 of Llama 2 7B Chat. A dictionary's best claim to monosemanticity and a behavior's best claim to linear separability therefore sit at opposite ends of the same stack of layers, a contrast only visible because this later paper supplied the second curve.