Residual stream — is the injection site whose growth constrains → Contrastive Activation Addition (CAA)

CAA's choice to inject into the residual stream, rather than into attention activations (ICV) or a sparse set of attention heads (ITI), is what makes a single normalization decision consequential across the whole method. The residual stream norm "grow[s] exponentially over the forward pass" (Heimersheim and Turner, 2023), which is why CAA normalizes steering-vector magnitudes to be comparable across behaviors but deliberately does not normalize across layers, in order to "preserve a 'natural norm' given the sampled activations" (contrastive-activation-addition, §"Vector normalization choices", p. 10). The authors flag the resulting confound directly: because they search for an optimal layer using one constant multiplier rather than sweeping multipliers per layer, a layer's apparent optimality is entangled with how large the residual stream's norm already is at that depth — the same multiplier represents a smaller relative perturbation at a late, high-norm layer than at an early, low-norm one (contrastive-activation-addition, §"Vector normalization choices", p. 10). So the residual stream is not a neutral injection point; its own growing scale shapes how CAA's layer-optimality results should be read.