Activation engineering / representation engineering (technique family)
Steering Llama 2 via Contrastive Activation Addition — inherited; Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
The broader family of alignment techniques, of which CAA is an instance, that steer language models by making targeted perturbations to their internal activations rather than through training; also called representation engineering.
None of the individual techniques for nudging a model's internal activations, rather than retraining it, is unique enough to have the whole field named after it alone. Collectively they go by activation engineering, or representation engineering: perturb the activations directly, and leave the training process exactly as it was.
Where CAA sits inside that umbrella comes next: one instance among several, alongside Activation Addition, Inference-Time Intervention, and In-Context Vectors, each making its own choice about where in the network to intervene and how the intervention gets derived in the first place.
Perturbing activations instead of retraining weights
Activation engineering, also called representation engineering, names a family of techniques that change a language model's behavior by directly modifying its internal activations during a forward pass, rather than by updating its weights through further training. The appeal is largely practical: no gradients, no training data beyond a handful of contrastive examples, and none of the risk of catastrophically disturbing other capabilities that large-scale finetuning can carry. The family shares one central object, the Steering vector — some direction in activation space taken to correspond to a behavior or concept — and its member techniques differ mainly in how that direction is extracted and where it is applied.
A short lineage, one technique at a time
The paper places itself within a specific 2022-23 lineage, citing Subramani et al. (2022) and Hernandez et al. (2023) as earlier instances alongside three it discusses at length: Activation Addition (ActAdd) builds a vector from a single prompt pair and injects it at one token position, working inconsistently across behaviors on GPT-2-XL alone; Inference-Time Intervention (ITI) narrows its search to a sparse set of attention heads found by linear probing; Representation Engineering (Zou et al. 2023) treats extraction itself, rather than steering, as the primary goal; and In-Context Vectors (ICV) intervenes across every layer's attention activations at once (contrastive-activation-addition, §"1 Introduction", p. 2). Contrastive Activation Addition (CAA) positions itself as this family's most broadly validated member: seven behaviors, RLHF-trained chat models, hundreds of contrast pairs per vector.
Practice ahead of theory
The paper is explicit about the field's evidentiary order: "beyond steering behaviors, work on activation engineering has also motivated a formalization of 'linear representation'" (contrastive-activation-addition, §"2 Related work", p. 3) — the Linear representation hypothesis arrived as an explanation for why this family of techniques kept working, not as a theory the family was built to test. Each accumulating empirical success, a single added or shifted direction producing a coherent behavioral change, needed an account, and Park et al.'s formalization is offered as that account after the fact — with CAA itself continuing the same pattern, a new technique cited as further motivation for the hypothesis it also relies on.