Sycophancy on NLP Survey / Sycophancy on Political Typology datasets
Steering Llama 2 via Contrastive Activation Addition — inherited
Two Anthropic evaluation datasets (from Perez et al., 2022) mixed together as the source of contrast pairs used to construct CAA's Sycophancy steering vector.
Sycophancy doesn't show up the same way corrigibility or survival instinct does; it's about whether a model bends its answer toward what it thinks the user wants to hear, which shows up most clearly in contexts like surveyed opinions and political positions rather than abstract hypotheticals about AI behavior.
The two Anthropic datasets behind that vector get their due here, one on NLP survey opinions, one on political typology, mixed together as the source of contrast pairs CAA uses to build its Sycophancy steering vector.
Two survey-style datasets, mixed together
Sycophancy on NLP Survey and Sycophancy on Political Typology are two of Anthropic's evaluation sets, both introduced in Perez et al. (2022) and mixed together as CAA's source of contrast pairs for the Sycophancy steering vector (contrastive-activation-addition, §"3.1 Sourcing datasets", p. 3). They are CAA's one departure from its two other sourcing routes: unlike Corrigibility, AI Coordination, Myopic Reward, and Survival Instinct, sycophancy's pairs don't come from the Advanced AI Risk evaluation dataset; unlike hallucination and refusal, they aren't freshly generated with GPT-4 either.
A different kind of contrast: opinions, not risk scenarios
Advanced AI Risk was written to probe dispositions like corrigibility and survival instinct that surface in hypothetical scenarios about AI deployment and control; the sycophancy datasets are built instead around opinions and survey questions, where a model can either state its own view or simply agree with whatever the human has already said. That domain mismatch has a downstream consequence: CAA also writes bespoke open-ended sycophancy questions rather than reusing these multiple-choice items directly for its open-ended evaluation, and cross-checks the resulting steering vector separately against TruthfulQA in Appendix H rather than treating it as validated by construction the way the four Advanced-AI-Risk-sourced behaviors are (contrastive-activation-addition, §"H Sycophancy steering and TruthfulQA", p. 14).
Part of the model-written evaluations program
Both datasets descend from Perez et al.'s broader model-written evaluations project, which used language models to generate large batches of behavioral test questions; sycophancy toward a stated political leaning or a claimed research finding is one disposition that program surfaces at scale. CAA imports this data wholesale rather than reproducing the generation pipeline, then builds its own diagnostic on top of it - see Sycophancy for how the resulting steering vector is used to test a specific hypothesis about why the behavior exists in the first place.