Steering grows out of inference time
A steering vector was originally something applied after training finished, a late correction bolted onto a model whose weights were already fixed. This theme follows what happens when the same vector moves earlier, into the training run itself, so that a model is pushed away from an unwanted trait while it is still being shaped rather than afterward.
The page contrasts fixing a trait after finetuning against preventing it during finetuning, and follows the second approach through its refinements, applying the push across several layers at once, and its rivals, a system prompt carrying the same message and a direct penalty on activation change that turns out not to work. A closing case study shows the preventative version blocking an unwanted trait without blocking the genuinely new facts the training was meant to teach.
CAA-style steering, in the 2023 paper, was an inference-time patch applied to an already-finetuned model. Steering-based mitigation of finetuning-induced persona shifts is this paper's umbrella framing for moving the same lever earlier, into training itself, and it names two strategies. Post-hoc steering keeps steering where CAA left it -- subtracting a scaled persona vector at each decoding step after finetuning -- and works, though large coefficients cost general capability on MMLU. Preventative steering instead amplifies the persona vector during finetuning, so the model feels less pressure to encode the trait in its weights in the first place; it preserves capability better than post-hoc steering and, combined with multi-layer steering -- applying a layer-incremental vector at every layer instead of one -- suppresses trait acquisition to near-baseline without undoing the finetuning's intended learning. The paper checks preventative steering against alternatives at both ends of the prompting-training spectrum: preventative prompting, a system prompt eliciting the trait prepended to every training sample, performs comparably at a moderate coefficient; the train-time regularization baseline, penalizing activation change along the trait direction directly, proves largely ineffective, since the model routes around the penalized direction; and CAFT, a prior concept-ablation method, works for evil and sycophancy but not hallucination. The fact-acquisition case study is the sharpest test of the whole approach: finetuning on 1,000 postdating facts predictably increases hallucination, and preventative steering suppresses that increase while, unlike inference-time steering, still letting the model learn the new facts.