CAA steering vector construction (Mean Difference over multiple-choice contrast pairs) — refines the single pair recipe of → Activation Addition (ActAdd)

caa-steering-vector-construction is explicitly modeled on Turner et al.'s Activation Addition but changes three things at once. First, the dataset: ActAdd derives a vector from the activation difference of a single pair of prompts, while CAA averages over hundreds of contrast pairs per behavior (up to 1000 for Hallucination and Sycophancy, per Appendix E), which "reduces noise in the steering vector, allowing for a more precise encoding of the behavior of interest" (contrastive-activation-addition, §"2 Related work", p. 2). Second, where the vector is applied: ActAdd adds its vector only to the first token position of other forward passes, while CAA adds it "to all and only token positions after the original prompt" (contrastive-activation-addition, §"2 Related work", p. 2). Third, scope of validation: ActAdd "does not consistently work for different behaviors, is not robust to different prompts, and was only tested on GPT-2-XL," whereas CAA is validated across seven alignment behaviors on Llama 2 7B and 13B Chat, models trained with RLHF (contrastive-activation-addition, §"2 Related work", p. 2). Each change targets a specific weakness of the single-pair recipe.