MMLU (Massive Multitask Language Understanding) — reruns the question behind → Alignment tax

InstructGPT named this cost in 2022: RLHF finetuning measurably regressed performance on public NLP datasets relative to the base GPT-3 model, a regression the paper called the alignment tax and tried to pay down with a pretraining-mix gradient term in PPO-ptx (instructgpt, §"4.2 Results on public NLP datasets", p. 14). CAA's MMLU table asks the same question of a different intervention: does making the model behave differently -- via activation steering rather than RLHF finetuning -- come out of general capability? The measured answer is close to zero: MMLU accuracy shifts by at most a few points under steering at either multiplier, for every tested behavior (contrastive-activation-addition, §"7 Effect of CAA on general capabilities", p. 6). Read against InstructGPT's framing, that is a meaningfully different result: RLHF's alignment tax was real enough to require a dedicated architectural fix; CAA's post-hoc, inference-time lever appears to leave the tax largely unpaid, which is one reason the paper frames steering as compatible with, rather than competing against, whatever capability cost the underlying RLHF training already incurred.