Train-time projection-regularization baseline
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — introduced
An alternative training-time mitigation the paper designs and tests: adding a regularization loss term that penalizes changes in the projection of activations along a trait's persona direction during finetuning. Found largely ineffective -- the model still expresses the trait post-finetuning -- likely because optimization pressure encodes the trait via alternative activation-space directions that bypass the penalized one.
Penalizing a model for moving toward an unwanted trait during training sounds like the more conservative, obvious fix — simpler than actively pushing activations around, and aimed at the exact quantity you want to control. The paper builds and tests this idea directly, and the obvious fix turns out not to work.
Adding the penalty to the finetuning loss is straightforward; why the model still ends up expressing the trait afterward, despite the added cost, is the more interesting part. The failure reveals something by contrast: discouraging movement along one known direction leaves the model free to reroute the same behavior through directions the penalty never touches, unlike the paper's preferred method, which removes the pressure to move at all.
What it is
The regularization baseline is a training-time mitigation the paper designs and tests as a natural alternative to steering, rather than one adapted from prior work: "a natural idea to suppress persona shift during finetuning is to add a regularization loss that penalizes changes in the projection of activations along trait-relevant directions" (persona-vectors, §"J.5 Train-time regularization along undesirable persona directions", p. 48). Unlike Preventative steering, which actively supplies activation movement along a persona direction, this approach discourages movement along that direction by adding a penalty term to the finetuning loss, an intuitive, minimal-intervention idea that targets the exact same projection Finetuning shift (activation-shift metric) measures.
Why it fails
The regularizer reduces the measured projection shift to some extent, but "the model is still able to express the trait after finetuning" (persona-vectors, §"J.5 Train-time regularization along undesirable persona directions", p. 48). The paper's diagnosis is that the next-token prediction loss "may unnaturally drive the model to encode the persona through alternative directions in activation space, thus bypassing the regularization signal" (persona-vectors, §"J.5 Train-time regularization along undesirable persona directions", p. 48). This is a Goodhart-shaped failure: the regularizer penalizes one measured proxy for trait presence, projection onto a single known direction, and gradient descent routes the same underlying prediction problem through a different, unpenalized direction instead, satisfying the letter of the constraint while still solving the training objective.
Why preventative steering avoids the same trap
Preventative steering does not merely discourage movement along the persona direction; during training it actively supplies that movement externally, at every step, via $h_\ell \leftarrow h_\ell + \alpha \cdot v_\ell$ (persona-vectors, §"5.2 Preventative steering limits behavioral shifts during finetuning", p. 8). That difference in mechanism, not merely in degree, is what the comparison is meant to isolate: regularization fights the symptom it can see, a single monitored projection value, while steering removes the underlying driver, the need to fit trait-eliciting training responses by changing internal representations at all, along any direction. Penalizing a measured direction and pre-empting the need for a direction turn out not to be equivalent interventions, even though both operate on the identical projection quantity.