A behavior becomes a direction in activation space
Ask where a behavior like refusal or sycophancy actually lives inside a language model, and the answer this theme gives is oddly literal: it lives along one direction in the model's activation space, and pushing the model's internal state along that direction, by a chosen amount and in a chosen sign, moves the behavior up or down like a dial. Getting from a vague tendency to a genuine, tunable direction is the whole trick, and it rests on a small piece of arithmetic rather than any retraining.
The page walks through how that direction gets built: averaging the activation difference between many closely matched pairs of answers, then adding the result back into the model at every position after the prompt, scaled by a signed multiplier. It also covers the guardrails around that core move, from normalizing vectors so different behaviors can be compared fairly, to the refinements the authors propose for narrowing where in the network the intervention lands.
A steering vector is a single direction in a model's activation space that, added to a forward pass, shifts output toward or away from a high-level behavior. Contrastive Activation Addition is the 2023 paper's method for building and applying one: the vector is added at every token position after the user's prompt, with a signed coefficient giving continuous control. CAA steering vector construction is the specific recipe, formalized as Equation 1: extract layer-L residual stream activations at the answer-letter token position for both completions in a dataset of paired prompts, then average their difference across the whole dataset. That averaging step applies the Mean Difference method, a pre-existing extraction technique the paper notes was also used by Zou et al. and Li et al. The multiple-choice contrast-pair format keeps the pairs clean: each pair is a question ending in answer letter A or B, so positive and negative prompts differ by exactly one token, validated by an appendix experiment showing the model justifies whichever letter it is conditioned on having chosen. The steering multiplier then doses the vector: sign selects direction, magnitude selects strength, though large magnitudes degrade text quality. Steering vector normalization standardizes magnitude across behaviors while deliberately leaving the residual stream's own layer-to-layer norm growth, the accumulating sum the vector is injected into, unnormalized. Targeted-token steering and steering outside the residual stream are the paper's own proposed refinements: narrowing the intervention to fewer token positions, or moving it to another point in the network, such as after the MLP, to better localize where behavior lives. Persona vectors are the 2025 paper's successor object: also a single direction in a model's residual-stream activation space, but keyed to a personality or character trait, such as evil, sycophancy, or hallucination, rather than to one of CAA's seven hand-picked behaviors, and computed the same way in spirit, as a difference in mean activations between trait-exhibiting and non-exhibiting responses. What makes it a successor rather than a rename is that it comes with its own automated, trait-agnostic extraction pipeline, so a vector for a newly named trait no longer needs a hand-built contrast dataset. Response-conditioned contrastive vector computation is that pipeline's construction recipe, playing the role CAA steering vector construction plays for CAA: for each extraction-set question it generates ten rollouts under a positive and a negative system prompt, filters them by trait expression score to keep only responses that clearly match their intended prompt, extracts residual-stream activations averaged over the response tokens rather than a single answer-letter token, and takes the mean-difference between trait-exhibiting and non-exhibiting activations at each layer, selecting whichever layer's vector steers most effectively. It reuses the same Mean Difference averaging step CAA's recipe applies, but over full free-form rollouts screened by an LLM judge instead of over matched multiple-choice pairs differing by one token.