Three levers on one model: prompting, finetuning, steering — adds a fourth lever after → InstructGPT's three-step training pipeline

InstructGPT's three-step pipeline was built to produce a finished, safety-tuned chat model; the 2023 paper's contribution is to treat that finished model as a substrate you can still act on. CAA is applied to Llama 2 Chat, itself "finetuned using RLHF for safety" through pretraining, supervised finetuning, and PPO against a learned reward model, the same three-stage recipe InstructGPT establishes (contrastive-activation-addition, §"1 Introduction", p. 2). Steering does not touch any of those three stages; it is a fourth lever, applied entirely at inference time after all of them have finished, and the paper's layer sweeps still find substantial, controllable movement in seven separate behaviors on that already-trained model (contrastive-activation-addition, §"4.1 Multiple-choice question datasets", p. 4). That a post-hoc, training-free intervention can still move behavior this much implies the finished pipeline's product is not as fixed as "three steps and done" suggests — there is slack in the trained model that the pipeline itself never used up.