Token-level cosine similarity analysis
Steering Llama 2 via Contrastive Activation Addition — introduced
An interpretability technique computing cosine similarity between a CAA steering vector and per-token residual stream activations during a forward pass, used to show that similarity with the vector tracks how strongly the target behavior is "present" at semantically relevant tokens.
A steering vector is a single direction in activation space, abstract and hard to inspect directly. But if that direction really encodes a behavior, it should light up at the specific words in a sentence where that behavior is actually happening, and stay quiet elsewhere.
The check itself gets walked through next: cosine similarity between a steering vector and each token's activations during generation, read against the actual text, with phrases like "I cannot help" lighting up for refusal and phrases about waiting for a bigger reward lighting up for myopia.
Measuring similarity, not intervening
Token-level cosine similarity analysis repurposes a steering vector as a passive measurement instrument rather than a control lever. During an ordinary forward pass -- no activation addition, no intervention at all -- the paper computes the similarity between a steering vector and the residual-stream activation at every individual token, using the standard cosine similarity $$\cos(\theta) = \frac{u \cdot v}{\|u\|\,\|v\|},$$ between the steering vector $u$ and each token's activation $v$. The resulting per-token score is checked against intuition: does it track how strongly the target behavior is "present" at that specific token, as a human reader would judge it (contrastive-activation-addition, §"8.1 Similarity between steering vectors and per-token activations", p. 7)?
Steering vectors as feature detectors
The scores line up with intuition closely enough to call the vectors feature detectors. On a refusal vector, the phrases "I cannot help" and "I strongly advise against" score positively, while text describing the harmful request itself, such as "hack into your friend's Instagram account," scores negatively (contrastive-activation-addition, §"8.1 Similarity between steering vectors and per-token activations", p. 7). On a Myopic Reward vector, tokens describing the model choosing a reward now score oppositely from tokens describing it choosing to wait for a larger reward later (contrastive-activation-addition, §"8.1 Similarity between steering vectors and per-token activations", p. 8). The analysis builds on the lead author's own earlier observation of the same effect (Panickssery, 2023a).
One object, two roles
A Steering vector is defined by what it does when added to activations: it shifts the output distribution toward or away from a behavior. This analysis shows the same object also works in reverse, with no addition at all -- merely projecting activations onto it recovers a signal about whether the behavior is currently active in the model's processing. A vector built to intervene on Refusal (as a steerable behavior) or myopia turns out to already encode enough about that behavior to recognize it token by token, which is part of the evidence, alongside Behavioral clustering, that CAA treats as support for the Linear representation hypothesis: the same linear direction that moves a concept when added also reads it out when merely measured against.