Response-conditioned contrastive vector computation
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — introduced
The pipeline's second stage: for each extraction-set question, generate 10 rollouts under both the positive and negative system prompts, filter rollouts by trait expression score (keeping only responses that clearly match their intended prompt), extract residual-stream activations averaged over response tokens, and take the mean-difference between trait-exhibiting and non-exhibiting activations at each layer, selecting the layer whose vector steers most effectively.
Having a set of contrastive prompts and questions still leaves the hard part undone: turning a model's free-form responses to those prompts into one clean numerical direction. Unlike a multiple-choice answer, a full paragraph can drift off-target in dozens of ways, so getting from raw generations to a trustworthy vector takes more than a simple average.
Rollouts get generated under contrasting instructions, filtered for ones that clearly match their intended condition, and averaged into a single mean-difference vector, with a layer sweep afterward picking the version that steers best. Set against the more tightly controlled multiple-choice format used in prior steering work, the contrast comes down to how much statistical cleanup free-form text demands that a fixed answer format never needed.
From artifacts to activations: generating and filtering rollouts
Persona-vector computation is the extraction pipeline's second stage, and it is where the artifacts the first stage produced actually get turned into numbers. For each of the 20 extraction-set questions, the pipeline generates 10 rollouts of the target model under the positive system prompt and 10 under the negative one — 400 raw responses per trait (persona-vectors, §"2.2 Extracting persona vectors", p. 3). Not every rollout actually displays the trait it was prompted toward — a model instructed to act "evil" can still produce a mild response — so each response is scored against the freshly generated rubric and only kept if it clearly matches its intended condition: trait score greater than 50 for a positive-prompted response, less than 50 for a negative-prompted one (persona-vectors, §"2.2 Extracting persona vectors", p. 3). Averaging in an off-target response at this stage would blur the eventual mean-difference signal with noise from a rollout that never actually diverged along the trait direction, which is why filtering happens before any activation is touched.
The mean-difference extraction, and where to read from
For every response that survives filtering, the pipeline extracts residual-stream activations at every layer $\ell$ and averages them over response tokens — a choice validated empirically rather than assumed: Appendix A.3 compares extracting from the final prompt token, an average over all prompt tokens, and an average over all response tokens, and finds response-token averaging "generally yields the strongest steering effect across traits" (persona-vectors, §"A.3 Extracting persona vectors at different token positions", p. 27). The persona vector at layer $\ell$ is then defined as "the difference in mean activations between responses that exhibit the trait and those that do not" (persona-vectors, §"2.2 Extracting persona vectors", p. 3). Adapting the per-response activation notation the paper defines formally in §6.1, $a_\ell(x,y)$ for the mean activation over response tokens at layer $\ell$ given prompt $x$ and response $y$, this is $v_\ell = \frac{1}{|R^+|}\sum_{(x,y)\in R^+} a_\ell(x,y) \;-\; \frac{1}{|R^-|}\sum_{(x,y)\in R^-} a_\ell(x,y)$, where $R^+$ and $R^-$ are the filtered sets of trait-exhibiting and trait-suppressing rollouts. No optimization or gradient descent is involved — the entire extraction is one subtraction of two averages, once the filtered contrast set exists.
Choosing a layer: a sweep, not a default
Computation does not commit to a layer in advance. Response-token activations are averaged into a candidate vector at every layer, and only afterward is "the most informative layer" selected by testing steering effectiveness across the full depth of the model, using the same steering coefficient at each candidate layer (persona-vectors, §"B.4 Selecting the most informative layer", p. 30). The result is not a single default: layer 20 is most informative for both evil and sycophancy on Qwen2.5-7B-Instruct, but layer 16 for hallucination on the same model, while layer 16 turns out to be most informative for all three traits on Llama-3.1-8B-Instruct (persona-vectors, §"B.4 Selecting the most informative layer", p. 30). The coefficient range swept also differs by model — up to 2.5 for Qwen, only up to 1.2 for Llama — before generation quality degrades, reflecting that how far a given layer's residual stream tolerates perturbation is a property of the model, not of the extraction method alone (persona-vectors, §"B.4 Selecting the most informative layer", p. 30).
A more general recipe than CAA's
Persona-vector computation and CAA's steering-vector construction are both instances of the same underlying Mean Difference recipe — collect two sets of contrasting activations and subtract their averages — but they manufacture their two-group split differently. CAA reads a single, fixed token position, the appended answer letter of a multiple-choice question, from a hand-built dataset of hundreds of (prompt, positive completion, negative completion) triples curated separately for each of seven behaviors (contrastive-activation-addition, §"3 Method", p. 3). Persona-vector computation instead reads an average over every token of a full free-form response, generated by the pipeline itself under a contrastive system prompt, and narrows the result with an LLM-judge filter (trait score above 50 / below 50) rather than trusting every generated pair by construction (persona-vectors, §"2.2 Extracting persona vectors", p. 3). Where CAA engineers a format that makes the contrast trivially clean — a single differing token needs no further filtering — persona-vector computation has to manufacture cleanliness statistically after the fact, because free-form rollouts vary in far more ways than an answer letter does.