The activation engineering family: from single pairs to datasets
No single paper invented the idea of steering a language model by editing its activations instead of its weights; it is a family of techniques that has kept growing for years, each trying a different recipe for turning a behavior into an editable direction, and, more recently, for moving that direction from something applied at inference time into something applied during training itself.
The page traces that lineage from its simplest form, a vector built from just one pair of prompts and added at a single position, through variants that search only a handful of attention heads or intervene at every layer at once, to the method this corpus centers on, averaged over hundreds of matched pairs, and on to a newer generation that automates the extraction step entirely and reaches into finetuning rather than stopping at generation.
Activation engineering, also called representation engineering, is the umbrella term for a family of 2022-25 techniques that steer or otherwise intervene on language models via their internal activations, most by perturbing them at inference time rather than by retraining the model, and the steering vector, a single direction encoding a behavior, is the object every member of the family produces, consumes, or, in the family's newest members, acts on during training itself. Activation Addition is the earliest and simplest instance: Turner et al. build a steering vector from the activation difference of just one pair of prompts and add it at the first token position, a method that works inconsistently across behaviors and was tested only on GPT-2-XL. Inference-Time Intervention narrows its search to a sparse set of attention heads, identified by linear probes trained on a contrastive question-answering dataset, then shifts activations along a Mean Difference vector at those heads alone to improve truthfulness. Representation Engineering, Zou et al.'s broader program, aims first at reading representations of concepts like honesty and emotion out of a model, testing Mean Difference among other extraction methods, before steering is treated as a downstream use. In-Context Vectors intervenes across every transformer layer's attention activations at once, rather than injecting at a single layer, to reduce toxicity and shift style. Contrastive Activation Addition is this theme's 2023 endpoint: it keeps ActAdd's approach of adding a difference vector back into activations and Zou's Mean Difference extraction, but replaces a single prompt pair or probed attention heads with an average over hundreds of tightly matched contrast pairs, differing by only one token, injected into the residual stream after the prompt across a broader range of behaviors. The family's newest generation, arriving in 2024 and 2025, pushes in two directions the earlier members didn't: automating extraction, and moving the intervention from inference into training. The automated concept-description-to-direction pipeline (Wu et al., 2025) automates the first step every earlier member did by hand, feeding an LLM a natural-language concept description and extracting a direction from the synthetic contrastive pairs it generates in response, the closest prior-work analogue to the 2025 persona-vectors paper's own extraction pipeline. The personality-trait vector space (Allbert et al., 2025) and emotion vectors (Dong et al., 2025) keep the family's Mean Difference extraction but widen what the direction targets, elicited personality traits across a 179-trait geometric space and five basic emotions respectively, rather than a single steering-vector behavior. Security vectors (Zhou et al., 2024) and CAFT (Casademunt et al., 2025) are the members that cross into training: security vectors pre-train detachable LoRA modules that are activated during finetuning to shield a model's weights from learning an undesired behavior and deactivated at inference to restore normal output, while CAFT zero-ablates a concept's activation-space projection during finetuning outright, with no tunable coefficient. Both anticipate the 2025 persona-vectors paper's own preventative steering; CAFT is in fact re-implemented and compared against it directly, found effective for evil and sycophancy, whose base-model projections are already negative so ablation acts like positive steering, but not for hallucination.