Steering Llama 2 via Contrastive Activation Addition

arXiv:2312.06681 · PDF · 19 pp. · its place among the papers

One paper, several tellings. The big picture states the paper’s own thesis and contributions in its own terms; Inside the paper follows the paper’s own arc; Across the corpus traces how it connects to the other papers; The concepts lists everything the paper uses, in reading order. Pick a tab, then use the +/− toggles to open it level by level, or set a granularity to read the whole thing at that zoom.

Every alignment technique on offer — RLHF, instruction finetuning, prompt engineering — works either by updating a model's weights or by hoping a prompt lands, and none of them says where inside the network a behavior like refusal or sycophancy actually lives. The paper's own claim is narrower and sharper than that: a behavior is a direction in the model's activations, and that direction can be found, dosed, and reversed without touching a single weight.

What follows is the paper's argument in its own vocabulary, at full altitude: the thesis that a behavior is a dosable direction, the construction that turns contrast pairs into one, the evidence across seven behaviors and two evaluation formats, what the method reveals about representation and about RLHF itself, and the limits the paper reports against its own result.

Read at

The 2023 argument: a behavior is a direction you can dose (5 chapters)

Evidence for the thesis runs on three legs: multiple-choice and open-ended evaluations showing all seven behaviors move in the intended direction; a capability floor, MMLU, that barely moves under the same intervention; and an interpretability chapter that reads the same vectors as measurements of where behavior lives and what RLHF changed. The paper reports its own limits alongside the result — an unoptimized finetuning baseline, a norm choice it flags against itself, GPT-4 rater noise — and closes by offering the same machinery as a red-teaming tool.

Chapter 1The thesis: a behavior is a direction you can dose (4 concepts)

The paper's central claim, stated plainly in its introduction and defended for the rest of its length, is that a high-level behavior — sycophancy, refusal, corrigibility — corresponds to a single direction in a model's activation space, discoverable from a dataset of paired prompts and controllable by adding that direction back into the residual stream with a signed multiplier. The premise licensing this is the linear representation hypothesis: that concepts this abstract still sit along linear directions rather than requiring some more tangled representation to describe them.

One direction, one behavior concept (4 connections)

A steering vector is a single direction in a model's activation space encoding a target behavior, such that adding it to a forward pass shifts output toward the behavior, and subtracting it shifts output away — the object the entire thesis rests on.

connections
The method the thesis is built to test concept (12 connections)

Contrastive Activation Addition is the paper's name for the specific method of building and applying a steering vector: extract it from contrast pairs, add it to the residual stream at every post-prompt token position, evaluated across seven behaviors on Llama 2 Chat.

connections
Why a vector should be enough concept (4 connections)

The linear representation hypothesis holds that high-level concepts are encoded as linear directions in an LLM's activation space — the theoretical premise that makes it plausible for a single added vector to shift a whole behavior rather than produce noise.

connections
Dosed, not just switched concept (3 connections)

The steering multiplier is what turns a direction into a controllable intervention: sign selects whether the behavior is encouraged or suppressed, and magnitude sets how strongly, though the usable range is capped by text-quality degradation at high magnitudes.

connections
Chapter 2The construction: contrast pairs to a vector (4 concepts)

Building the vector takes three decisions, stated in Section 3. Contrast pairs are multiple-choice questions differing by exactly one token — the answer letter — so nothing else can leak into the difference. The Mean Difference method averages that difference across a whole dataset rather than trusting any single pair. And the resulting vector is injected into the residual stream, the running sum every layer writes into, at every token position after the prompt, so its effect compounds across the entire generated response.

A single token separates positive from negative concept (2 connections)

Every contrast pair is a multiple-choice question ending in answer letter A or B, with the positive and negative prompt otherwise identical — a design meant to cancel every confound except the one variable the paper wants to isolate.

connections
  • Multiple-choice contrast-pair formatcancels the confounders in → CAA steering vector construction (Mean Difference over multiple-choice contrast pairs)
  • Multiple-choice contrast-pair formatrepurposes the ab mechanic of → Multiple-Choice Comparison Format hindsight
The averaging step concept (3 connections)

Mean Difference extraction averages the activation difference between positive and negative completions across an entire dataset of pairs, rather than trusting the noise in any single comparison — the step that turns hundreds of individually noisy differences into one usable direction.

connections
Equation 1, in full concept (6 connections)

Formalized as a single equation: for a dataset of (prompt, positive completion, negative completion) triples, take the layer-L activation difference at the answer-letter position for each triple, and average across the dataset — the complete recipe for a CAA steering vector.

connections
  • CAA steering vector construction (Mean Difference over multiple-choice contrast pairs)refines the single pair recipe of → Activation Addition (ActAdd)
  • Inter-layer steering vector similarity and transfer — demonstrates the generality ofCAA steering vector construction (Mean Difference over multiple-choice contrast pairs)
  • Mean Difference (MD) vector extraction — supplies the extraction rule forCAA steering vector construction (Mean Difference over multiple-choice contrast pairs)
  • Multiple-choice contrast-pair format — cancels the confounders inCAA steering vector construction (Mean Difference over multiple-choice contrast pairs)
  • PCA visualization of contrastive activations — screens datasets forCAA steering vector construction (Mean Difference over multiple-choice contrast pairs)
  • Response-conditioned contrastive vector computation — generalizes beyond the answer letter recipe ofCAA steering vector construction (Mean Difference over multiple-choice contrast pairs) hindsight
Where the vector is added back in concept (4 connections)

The residual stream is the running sum of every layer's output that a transformer's computation flows through and accumulates in — the site CAA chooses to inject its vector into, rather than attention activations or a searched set of attention heads.

connections
Chapter 3The evidence: seven behaviors move, generalization holds (4 concepts)

Two evaluation regimes carry the paper's evidence. Multiple-choice behavioral evaluation sweeps every layer at fixed multipliers and finds all seven tested behaviors move in the intended direction, with a consistent optimal layer across behaviors. Open-ended generation evaluation repeats the test in free text, scored by GPT-4 against a rubric written per behavior, and the effect survives leaving the contrived multiple-choice format. A third piece of evidence is negative: MMLU, run under every steering condition, shows no significant change, which the paper reads as its capability floor holding.

Every one of seven, in the intended direction concept (2 connections)

Multiple-choice behavioral evaluation sweeps a steering vector across every layer at multipliers of -1 and +1, scoring effect as the average token probability given to the behavior-matching answer on 50 held-out questions — and finds a consistent optimal layer where every tested behavior moves as intended.

connections
The result survives leaving multiple-choice concept (4 connections)

Open-ended generation evaluation applies the same steering vectors to free-form answers rather than contrived A/B questions, testing whether the effect generalizes beyond the format it was extracted from — necessary, the paper argues, for the method to be useful at all.

connections
The instrument doing the scoring concept (2 connections)

GPT-4 rates every open-ended response 1-10 against a rubric written per behavior, standing in for a human evaluator at a scale hand-scoring could not match.

connections
  • GPT-4authors → Custom refusal contrastive dataset
  • GPT-4rates outputs for → Open-ended generation evaluation with GPT-4 rating
And general capability barely moves concept (4 connections)

MMLU, 57 subjects of multiple-choice general knowledge, is run under every steering condition unmodified, and the paper reports no significant change to the model's average probability of the correct answer — the negative result that lets the positive results stand without an asterisk.

connections
Chapter 4What it reveals: emergence and what RLHF changed (4 concepts)

Beyond controlling behavior, the paper's own techniques double as findings about the model. Behavioral clustering shows a targeted behavior separating linearly in activation space suddenly, around one-third of the way through the network's layers, rather than emerging gradually — direct evidence for the linear representation hypothesis the method assumes. Token-level cosine similarity turns the same vector into a passive reader, lighting up at semantically relevant tokens with no intervention at all. And comparing vectors built from Llama 2's base and RLHF-tuned chat variants finds them unusually similar in the middle layers, which the paper reads as a measurement of RLHF itself: it reweights which behaviors get expressed without much rewriting how they're represented internally.

Behavior separates suddenly, not gradually concept (3 connections)

Behavioral clustering is the finding that a contrastive dataset's activations separate by whether the model's answer matches the target behavior, and that this separation appears abruptly around one-third of the way through the network's layers rather than building up gradually.

connections
A passive detector for free concept (1 connection)

Token-level cosine similarity between a steering vector and ordinary per-token activations, computed with no intervention at all, tracks how strongly the target behavior is present at each token — the same object built to control turns out to also detect.

connections
What one training stage did and didn't do concept (2 connections)

Comparing steering vectors generated from Llama 2's base model against Llama 2 Chat finds elevated similarity specifically between layers 7 and 15, and finds base-derived vectors still steer the chat model — evidence that RLHF changes which behaviors surface more than it changes how they're represented.

connections
The stage being read, not modified concept (6 connections)

RLHF is the training paradigm that produced Llama 2 Chat from Llama 2 base; the paper never modifies it, only uses the base/chat pair it produced as an instrument for reading what that one training stage left behind internally.

connections
Chapter 5The limits and the offer (4 concepts)

The paper reports its own weak points rather than leaving them implicit. The comparison to finetuning uses one fixed, unoptimized set of hyperparameters, so the finetuning baseline may understate what that lever can do. The comparison to system-prompting likewise searches a limited set of prompts. A vector-normalization choice — standardizing magnitude across behaviors but not across layers — is flagged as a possible confound in the paper's own layer-optimality results. Against these limits, the paper closes by offering its method a new job: using CAA as an adversarial red-teaming tool, deliberately eliciting an unwanted behavior to test whether training actually removed it or only suppressed it.

An unoptimized baseline concept (2 connections)

The finetuning comparison uses one fixed set of hyperparameters — one epoch, plain SGD, no tuning of learning rate or loss function — which the paper itself flags as likely understating what a better-optimized finetuning baseline could achieve.

connections
A prompting baseline, not exhaustively searched concept (1 connection)

The system-prompting comparison tests a handful of hand-written prompts rather than an exhaustive search, which the paper concedes means a more carefully optimized prompt might close some of the gap it reports in CAA's favor.

connections
A norm choice the paper flags against itself concept (1 connection)

Steering vector normalization standardizes magnitude across behaviors but deliberately leaves each layer's own norm unnormalized, to preserve the residual stream's natural growth — a choice the paper admits could skew its own layer-optimality results, since a fixed multiplier is a smaller relative push at a high-norm layer than at a low-norm one.

connections
The offer: turn the same lever into an audit concept (4 connections)

CAA as an adversarial red-teaming tool is the paper's proposed extension: use the identical steering method to elicit an unwanted behavior rather than suppress it, on the premise that a behavior easily triggered by a small internal perturbation is plausibly reachable in deployment too.

connections
  • CAA as an adversarial red-teaming tool (proposed application)skips the prompt search of → Automated Red Teaming hindsight
  • CAA as an adversarial red-teaming tool (proposed application)inverts the purpose of → Contrastive Activation Addition (CAA)
  • Jailbreaks — motivatesCAA as an adversarial red-teaming tool (proposed application)
  • CAA as an adversarial red-teaming tool (proposed application)relocates the attack surface of → Red Teaming hindsight