Inter-layer steering vector similarity and transfer
Steering Llama 2 via Contrastive Activation Addition — introduced
An analysis of how similar steering vectors generated at different layers are to each other (finding higher similarity between nearby layers, with slower decay in the model's later half), paired with a test of whether a vector extracted at layer 13 still steers behavior when applied at other layers.
If a steering vector genuinely captures a behavior rather than some quirk of one particular layer, it shouldn't matter too much exactly where in the network it's extracted or applied. Two questions follow from that: do vectors built at nearby layers actually resemble each other, and does a vector extracted at one layer still work when it's added somewhere else?
Both halves of that check get reported ahead: similarity between layers fades with distance but more slowly in the back half of the network, and a vector taken from layer 13 keeps steering behavior when injected elsewhere too, up to a point, where the effect drops off sharply around layer 17.
How similar are vectors from nearby layers?
This analysis asks how a steering vector for the same behavior changes as it is extracted from different layers of the network, using cosine similarity between vectors built at different depths (Figure 7, comparing Myopic Reward vectors from Llama 2 7B and 13B Chat). The pattern is consistent: "vectors from closer layers have a higher similarity. This similarity diminishes for more distant pairs of layers... the rate of similarity decline is slower in the latter half of the model" (contrastive-activation-addition, §"8.2 Similarity between vectors generated at different layers", p. 7). The authors' interpretation is that once the model extracts the high-level information needed to describe an abstract concept, the representation "converges" and remains more consistent across subsequent layers, rather than continuing to change.
Does a layer-13 vector work elsewhere?
A second experiment tests whether a vector's usefulness is tied to the layer it came from. A vector built at layer 13 -- the layer CAA steering vector construction (Mean Difference over multiple-choice contrast pairs) settles on for the 7B model -- is applied at other layers instead, and the steering effect still holds; for some earlier layers it is even larger than at layer 13 itself, before there is "a steep drop-off in effect size around layer 17" (contrastive-activation-addition, §"8.2 Similarity between vectors generated at different layers", p. 7). Past that point, the authors suggest, "relevant information on abstract representations has been used for further processing and can no longer be manipulated in the same way" (contrastive-activation-addition, §"8.2 Similarity between vectors generated at different layers", p. 7).
Generality as evidence, with a limit
Together, the two findings argue that what CAA steering vector construction (Mean Difference over multiple-choice contrast pairs) extracts at a fixed layer is closer to a general, portable representation of the behavior than a layer-13-specific artifact: nearby layers agree with each other, and the vector keeps working when moved. That supports the same claim Behavioral clustering and Token-level cosine similarity analysis make from different angles -- that these directions correspond to something the model represents in a fairly stable, layer-independent way once the concept has been computed, consistent with the Linear representation hypothesis. The layer-17 cliff marks where that generality runs out: past it, the representation appears to have already been consumed by further processing and is no longer freely manipulable via the same direction.