Steering-based mitigation of finetuning-induced persona shifts
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — introduced
The paper's umbrella framing for two steering-based strategies to counteract undesirable persona shifts caused by finetuning: post-hoc steering (inhibiting the persona vector after finetuning) and preventative steering (amplifying the persona vector during finetuning). Both are shown to outperform prompt-based mitigation baselines.
Once finetuning has pushed a model's character somewhere unwanted, there's a real choice about when to step in: fix it after the fact, once the damage is visible, or head it off before training ever produces it. Steering-based mitigation is the paper's umbrella term for its two answers to that choice, both built from the identical underlying tool.
Both approaches share one mechanism, distinguished only by sign and timing, and both are held to the identical coherence and capability checks before their results count. Extended experiments beyond the headline comparison round things out: steering in either direction can suppress an unwanted trait shift without undoing whatever a finetuning run was actually meant to teach.
What unifies the two approaches
Steering-based mitigation is the paper's umbrella framing for §5, introduced once §4 has established that finetuning-induced persona shifts are mediated by movement along persona directions: "we show that persona shifts can be mitigated by steering along the associated persona vector. We explore two primary approaches: (1) inhibiting the persona vector after finetuning; and (2) amplifying the persona vector during finetuning" (persona-vectors, §"5 Steering can mitigate finetuning-induced persona shifts", p. 7). The first approach is Post-hoc (inference-time) steering mitigation, applied to an already-finetuned model at inference time; the second is Preventative steering, applied during the finetuning run itself. Both intervene on the identical object, a persona vector at a chosen layer, with only sign and timing distinguishing them, and both are explicitly framed as answers to the same problem the section opens with, rather than as a single technique wearing two names.
The evaluation floor both methods share
Both mitigations are validated against a coherence score floor before their headline trait-suppression numbers are reported at all, guarding against the degenerate outcome where a steering coefficient drives a trait score toward zero simply by breaking the model into incoherent text. The floors differ by method: post-hoc steering results require "average response coherence... above 75" (persona-vectors, §"5.1 Post-hoc steering mitigates behavioral shifts", p. 7), while preventative steering's reported results maintain "an average coherence score across all models above 80" (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8), a stricter bar for the training-time method. Neither method's own results section states this asymmetry; it is only visible by reading both thresholds side by side. General capability is checked separately via MMLU accuracy, giving the section two independent instruments, coherence for fluency and MMLU for competence, neither of which a coefficient that merely suppresses trait-relevant vocabulary could satisfy by accident.
What the extended experiments establish
Appendix J extends both methods well past the two headline comparisons in §5. Steering, in either direction, suppresses general persona shift without reversing the domain-specific behavior a finetuning run was meant to teach, tested on EM-style datasets where the intended learning (for example reproducing GSM8K-style mistakes) and the unwanted persona drift are cleanly separable (persona-vectors, §"J.1 Steering preserves domain-specific learned behavior while suppressing general persona shift", p. 45). Both variants also outperform prompt-based mitigation: a negative system prompt generated by the same extraction pipeline suppresses trait expression less effectively than steering at a coefficient chosen to keep coherence above 80 (persona-vectors, §"J.2 Comparison with system prompt-based mitigation", p. 45). And preventative steering specifically, applied to a benign dataset that would not have shifted the persona in the first place, leaves trait scores and MMLU accuracy essentially unchanged, evidence the intervention does not simply degrade the model by default (persona-vectors, §"J.6 Assessing side effects of preventative steering on benign data", p. 48).