Activation Addition (ActAdd)
Steering Llama 2 via Contrastive Activation Addition — inherited; Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
A prior steering technique (Turner et al., 2023) that generates a steering vector from the activation difference of a single pair of prompts and adds it at the first token position of other forward passes; tested only on GPT-2-XL. CAA is directly modeled on and contrasted with this method.
Before CAA, there was a simpler version of the same idea: take one pair of prompts, one showing a behavior and one not, subtract their activations, and add the result back in at the very first token of a new generation. It worked, sometimes, on GPT-2-XL, but a single pair of prompts carries a lot of incidental noise along with whatever behavior it's meant to capture.
Activation Addition is the direct ancestor CAA is modeled on and departs from, and the departures are specific: an average over hundreds of prompt pairs instead of one, and a vector injected across every token position after the prompt instead of just the first.
A steering vector from one prompt pair
Activation Addition (ActAdd) is the earliest technique in the Activation engineering / representation engineering (technique family) family and the one Contrastive Activation Addition (CAA) is most directly modeled on. Turner et al. (2023) build a single steering vector by running a model on one pair of prompts -- one exhibiting a behavior, one not -- extracting the intermediate activations of both at a chosen layer and token position, and taking their difference. That difference vector is then added back into the residual stream, but only at the first token position of subsequent forward passes, nudging the model's completions toward or away from the behavior. It is a minimal recipe: no dataset, no averaging, just one contrast and one injection point. The paper is explicit about where this minimalism breaks down: ActAdd "does not consistently work for different behaviors, is not robust to different prompts, and was only tested on GPT-2-XL" (contrastive-activation-addition, §"2 Related work", p. 2), a much smaller and non-RLHF-trained model than the Llama 2 Chat models CAA targets.
What CAA keeps and what it changes
CAA states plainly that "our technique is similar to Activation Addition" (contrastive-activation-addition, §"2 Related work", p. 2) -- both add a difference-of-activations vector back into the forward pass -- but it changes three things at once. First, the source of the vector: instead of one prompt pair, CAA steering vector construction (Mean Difference over multiple-choice contrast pairs) averages over hundreds of contrast pairs per behavior, which "reduces noise in the steering vector, allowing for a more precise encoding of the behavior of interest" (contrastive-activation-addition, §"2 Related work", p. 2). Second, where the vector is applied: rather than only the first token position, CAA adds it "to all and only token positions after the original prompt" (contrastive-activation-addition, §"2 Related work", p. 2). Third, the breadth of validation: seven alignment-relevant behaviors across two RLHF-trained chat models, rather than one model and inconsistent results. Each change targets a specific place where the single-pair recipe was fragile.
One entry in a larger family
ActAdd sits alongside Inference-Time Intervention (ITI), Representation Engineering (Zou et al. 2023), and In-Context Vectors (ICV) as instances of the same underlying move: perturb a model's activations directly, rather than retrain it, to shift a targeted behavior. The paper's introduction names them together as "the set of alignment techniques known as 'activation engineering' or 'representation engineering'... work by making targeted perturbations to a model's activations" (contrastive-activation-addition, §"1 Introduction", p. 2). All four produce or consume a Steering vector. What distinguishes ActAdd within that family is chronological and methodological priority: it is the simplest possible version of the idea, against which every later refinement -- CAA's dataset averaging, ITI's head-localization, RepE's broader representation-reading program -- can be read as a fix to a specific limitation of the single-pair recipe.