Steering vector
Steering Llama 2 via Contrastive Activation Addition — inherited; Persona Vectors: Monitoring and Controlling Character Traits in Language Models — refined
A single direction in a model's activation space that encodes a high-level behavior or concept, such that adding it to the activations (or subtracting it) shifts a model's outputs toward (or away from) that behavior — whether injected during a single forward pass at inference time or, in later work, during the training updates themselves. The object at the center of activation engineering: prior work extracts steering vectors from single prompt pairs, optimization, or probes; CAA extracts them by averaging over datasets of contrast pairs; persona vectors extract them via an automated pipeline and reuse them as both inference-time and training-time controls.
A single direction in a model's activation space, added or subtracted during a forward pass, turns out to be enough to shift what that model outputs toward or away from a target behavior. That's a striking amount of sway for something so simple, and different techniques disagree about exactly how to find the direction in the first place, and, more recently, about when it should even be applied.
The steering vector sits at the center of this corpus as the object several pages are built around, tracking how early work extracted directions from single prompt pairs, from optimization, or from trained probes, before showing how one paper's averaged, dataset-wide version fits into that lineage. A later paper narrows the same object to personality traits specifically and puts it to two further jobs beyond nudging a forward pass, reading it to predict behavior before generation and injecting it during training rather than only after.
A direction that stands in for a behavior
In a transformer's activation space, a steering vector is a single fixed direction — one vector shaped like a layer's hidden state — hypothesized to correspond to some coherent, high-level property of the model's behavior: sentiment, honesty, refusal, sycophancy. The idea rests on treating activation space as though many such properties are encoded roughly linearly, so that moving a model's activations along the right direction during a forward pass shifts its output distribution toward or away from the corresponding property, in a way an arbitrary random direction would not. This is the object Activation engineering / representation engineering (technique family) as a field is organized around: every technique in that family either extracts a steering vector from examples, applies one during inference, or both.
How CAA builds one, against how earlier work did
The term and the underlying object predate this paper — Subramani et al. (2022) called them latent steering vectors, and Turner et al.'s Activation Addition (ActAdd) built one from the activation difference of a single pair of prompts, added only at the first token position. Contrastive Activation Addition (CAA)'s version is built by CAA steering vector construction (Mean Difference over multiple-choice contrast pairs) using Mean Difference (MD) vector extraction averaged over hundreds of Multiple-choice contrast-pair format pairs rather than one, then injected into the Residual stream at every token position after the prompt with a chosen Steering multiplier (contrastive-activation-addition, §"Abstract", p. 1). Inference-Time Intervention (ITI) is a related but narrower consumer of the same kind of vector, applying it to a sparse set of attention heads rather than the whole stream.
A vector that also works as a detector
The paper treats its own steering vectors as objects worth reading, not only applying. Token-level cosine similarity analysis finds a vector doubles as a passive detector: taking its dot product against per-token activations during ordinary generation, with no intervention at all, produces a signal that tracks how much of the behavior is "present" at each token — refusal phrases like "I cannot help" and "I strongly advise against" score positively on the refusal vector, while a harmful request scores negatively (contrastive-activation-addition, §"8.1 Similarity between steering vectors and per-token activations", p. 7). That a vector built to push behavior in one direction also recognizes the behavior when merely observed is offered as evidence consistent with the Linear representation hypothesis this whole approach leans on.
2025: from a per-token nudge to a training-time lever and a training-data auditor
Persona Vectors keeps the same underlying object — a single direction added to or subtracted from residual-stream activations — but pushes what a steering vector is used for well past the inference-time nudge CAA introduced it as. Post-hoc steering applies the familiar update rule at every decoding step of an already-finetuned model, h_ℓ ← h_ℓ − α·v_ℓ, to suppress a persona trait a finetuning run introduced (persona-vectors, §"5.1 Post-hoc steering mitigates behavioral shifts", p. 7). Preventative steering moves the same intervention earlier, adding the vector during the finetuning forward passes themselves, so the model never has to shift its own weights toward the undesired direction to fit the training data in the first place — "relieving the model of the need to shift in that direction" (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8). This is a genuinely new role for the object: not a patch applied to a fixed model at generation time, but a regularizer that participates in the gradient step, and extending it across multiple layers makes it more effective still at holding a trait near baseline without added MMLU cost (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 9).
The vector's second new role is as a reading instrument, extended past the token-level detector CAA already demonstrated. Rather than reading per-token activations during generation, projecting a prompt's final-token activation onto the vector predicts the trait expression of a response not yet produced (r = 0.75–0.83 across system-prompt and many-shot elicitation) (persona-vectors, §"3.3 Monitoring prompt-induced persona shifts via projection", p. 5); projecting the difference between a finetuned and base model's activations onto the vector yields the Finetuning shift (activation-shift metric), a scalar summary of how far training moved the persona (persona-vectors, §"4.2 Activation shift along persona vector predicts trait expression", p. 7); and projecting candidate training responses onto the vector before any finetuning happens, the Projection difference (pre-finetuning data-screening metric), forecasts which datasets and samples will induce the shift (persona-vectors, §"6.1 Predicting post-finetuning behaviors from data", p. 9). Three distinct readings — of a prompt, of a weight update, and of a dataset — all reuse the identical dot product CAA first showed could double as a passive detector.