Preventative steering
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — introduced
A novel training-time mitigation method that proactively steers the model toward the undesired persona direction during finetuning, relieving the model of pressure to encode that trait in its weights and thereby 'canceling out' the objective's pull toward the trait. Compared with post-hoc steering, it better preserves general capability (MMLU) and, with multi-layer application, can suppress trait acquisition to near-baseline even on datasets designed to elicit it, without reversing the finetuning's intended domain-specific learning.
Correcting a model after finetuning has already reshaped its weights comes at a cost — suppressing a trait after the fact tends to dull general capability along with it. This method tries a different timing entirely: intervene during training itself, so the model never has to encode the unwanted trait into its weights in order to fit the training data.
Injected into the forward passes finetuning itself relies on, the intervention is traced from what it achieves relative to the after-the-fact alternative through where a single layer still falls short. Comparisons against other training-time fixes round out the account, ending with its most demanding trial: a case where the very thing being suppressed is entangled with genuinely useful new knowledge.
How it fits
Preventative steering is the paper's own novel contribution among its mitigation methods, the other half of the steering-mitigation pair and the one built specifically for this paper rather than adapted from prior work. It sits in a small family of training-time interventions on internal activations that the paper explicitly positions itself against: "recent work has proposed that intervening on internal activations during finetuning can be effective for controlling resulting generalization" (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8), citing Casademunt et al. (2025)'s CAFT, which zero-ablates a concept direction during training, and, in a footnote, Zhou et al. (2024)'s pre-trained "security vectors": detachable LoRA modules activated during finetuning to shield a model from harmful updates, then disabled for inference (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8, n. 7). Preventative steering answers the same problem these methods address, using a directional, signed intervention rather than either's ablation or LoRA-shielding mechanism.
The mathematics
Preventative steering applies the mirror image of Post-hoc (inference-time) steering mitigation's update rule, but during finetuning's forward passes rather than at inference, and with the sign reversed, adding rather than subtracting the scaled persona vector:
$$h_\ell \leftarrow h_\ell + \alpha \cdot v_\ell$$
injected into the residual stream at every training step, at layer $\ell$, throughout finetuning (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8). Because this happens inside the forward pass that produces the logits the finetuning objective is computed against, the model's weights are updated by ordinary backpropagation through an already-shifted activation: the finetuning run still uses standard next-token prediction loss on the training responses, following one epoch of rank-32 rsLoRA at learning rate $10^{-5}$ (persona-vectors, §"D.3 Finetuning details", p. 38), but that loss is computed on top of activations the intervention has already pushed toward the undesired direction. Elsewhere the paper attributes exactly this dynamic, the next-token prediction loss driving the model to encode a trait through whatever activation-space route remains available, as the reason a related regularization approach fails (persona-vectors, §"J.5 Train-time regularization along undesirable persona directions", p. 48); preventative steering exploits the same loss context rather than fighting it, by supplying the movement externally so the objective has less pressure to reproduce it through weight changes at all. The paper describes the resulting effect in its own words: "we proactively steer the model toward the undesired persona direction during training, relieving the model of the need to shift in that direction to fit the training data. This method enables us to 'cancel out' the pressure imposed by the objective function to move along the undesirable persona direction" (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8).
What it achieves, and where a single layer falls short
Across datasets, preventative steering "effectively reduces training-induced persona shifts, while also maintaining an average coherence score across all models above 80," and "better preserves the model's general capabilities compared to inference-time steering, as measured by MMLU accuracy" (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8), the paper's central capability-preservation claim, and the reason preventative steering is presented as the preferred fix where it applies. But single-layer preventative steering "does not always fully prevent trait acquisition, particularly for datasets that are intentionally designed to encourage that trait" (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8). The fix is Multi-layer steering with layer-incremental vectors, which applies a layer-incremental vector $\vec{v}_\ell^{\text{incremental}} = \vec{v}_\ell - \vec{v}_{\ell-1}$ at every layer to avoid double-counting overlapping directional signal; applied throughout training, it pushes trait acquisition down to near-baseline levels even on the hardest datasets, with no additional MMLU cost (persona-vectors, §"J.3 Steering across all layers", p. 46).
Comparisons against other training-time interventions
Against CAFT, using the paper's own persona directions, CAFT works well for evil and sycophancy but is far less effective for hallucination, because evil and sycophancy already have negative average projection in the base model, so zero-ablating them acts like positive preventative steering by construction, while hallucination's near-zero base projection leaves CAFT with little to erase (persona-vectors, §"J.4 Comparison with CAFT", p. 47). Preventative steering also outperforms a natural do-it-yourself alternative the paper builds and tests itself, the regularization baseline: penalizing projection change during finetuning reduces the shift somewhat, but the model still expresses the trait afterward, because optimization pressure finds it cheaper to route the trait through activation directions the loss term never penalizes (persona-vectors, §"J.5 Train-time regularization along undesirable persona directions", p. 47-48). A prompt-space analog, Preventative prompting, reproduces much of the same effect at a moderate coefficient without touching activations at all, showing the underlying "vaccinate the training signal" idea does not strictly require steering, even though steering remains more flexible because a coefficient is continuously tunable in a way a system prompt's strength is not.
Stress tests and what it does not cost
The fact-acquisition case study is the paper's most adversarial test of preventative steering's promise: finetuning on 1,000 facts postdating the model's training cutoff substantially raises hallucination, and a coefficient of 1.25 is needed to bring it back to the base model's baseline of roughly 20, yet new-fact accuracy drops only slightly at that coefficient, and MMLU stays essentially unchanged, while inference-time steering "tends to break the model" outright on the same task (persona-vectors, §"J.7.1 Preventative steering on a fact-acquisition task", p. 48-49). Applied to a benign dataset that would not have shifted the persona anyway, preventative steering likewise has "only a negligible effect" on trait scores and MMLU (persona-vectors, §"J.6 Assessing side effects of preventative steering on benign data", p. 48). Read against InstructGPT's earlier finding that RLHF alignment training measurably costs public-benchmark accuracy, the original alignment tax (instructgpt, §"4.2 Results on public NLP datasets", p. 14), preventative steering's near-zero MMLU cost answers the same cost-accounting worry InstructGPT first raised, for a mechanism its authors could not have anticipated; persona-vectors never invokes InstructGPT or the term "alignment tax" itself, so the connection is only visible in hindsight.