The public NLP benchmark suite used to catch regressions
To find out what a training or steering intervention costs, you need instruments that were never built to measure alignment at all. This theme covers the standard academic NLP benchmarks, reading comprehension, entailment, coreference, and the rest, that InstructGPT (2022) runs its models against to catch anything RLHF might have quietly broken, plus a broad knowledge test a 2023 steering paper borrows for the same purpose.
A drop on any of these relative to plain GPT-3 is exactly the evidence InstructGPT's authors need for what they call the alignment tax; the 2023 paper repeats the same logic in a different key, checking whether its steering vectors move a model's score on that knowledge test at all, and finding they mostly do not.
To measure the alignment tax RLHF imposes on general language ability, InstructGPT (2022) runs its models against a battery of standard academic understanding benchmarks it did not design, and Steering Llama 2 via Contrastive Activation Addition (2023) later reaches for the same kind of instrument to check a different intervention. DROP tests discrete reasoning such as counting and sorting over paragraphs; SQuAD v2 adds unanswerable questions to reading comprehension; QuAC evaluates conversational question answering; HellaSwag requires selecting the most plausible continuation of a scenario; SST is a binary sentiment classifier; and RTE and WSC, both drawn from the SuperGLUE suite for general-purpose language understanding, test textual entailment and pronoun coreference resolution respectively. MMLU, a battery of multiple-choice questions spanning 57 subjects across science, technology, humanities, and social sciences, is the benchmark the 2023 paper adds to this suite: it reports the model's average probability of the correct answer under positive and negative steering multipliers and finds no significant change, evidence that CAA's activation-space intervention leaves general capability intact even as it moves a targeted behavior. None of these benchmarks were built with instruction-following, human preference, or activation steering in mind; they exist to test comprehension and inference competence in the abstract. Their role in this corpus is specifically evaluative: a drop in performance on any of them is evidence that an intervention traded away some general capability for its targeted gains, exactly the tradeoff the alignment tax names for InstructGPT's RLHF fine-tuning, and the capability floor MMLU is checked against for CAA's steering vectors.