Preventative prompting
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — introduced
A prompt-based analog to preventative steering, tested in the fact-acquisition case study: a system prompt eliciting the undesired trait is prepended to every training sample, reducing the model's need to update its weights toward that trait. Performs comparably to preventative steering at a moderate coefficient, though steering offers a wider, more flexible control range.
An intervention that works by nudging a model's activations raises an obvious question: could the same protective effect come from something far simpler, like just telling the model what not to become, throughout training rather than only afterward? Preventative prompting is that text-based version, tested inside the paper's one case study built around a trait entangled with genuinely useful knowledge.
Prepending an instruction to every training example matches the activation-based method at a moderate setting, on every metric tracked. Where the comparison breaks down matters more: a written instruction has no clean way to be turned up further, while the activation-based version can simply be pushed harder for a stronger effect.
What it is
Preventative prompting is a prompt-space analog to Preventative steering, introduced specifically within the fact-acquisition case study rather than as a headline method: instead of injecting a persona vector into activations during training, a system prompt eliciting the undesired trait is prepended to every training sample (persona-vectors, §"J.7.2 Preventative prompting on a fact-acquisition task", p. 49). It runs on the identical underlying logic as preventative steering: proactively supplying the undesired behavior during training "cancels out" the pressure that would otherwise push the weights toward that behavior, but through text conditioning rather than an activation-space intervention.
How it compares to preventative steering
On the fact-acquisition task, preventative prompting matches preventative steering at a moderate coefficient (0.5) across all three tracked metrics -- hallucination score, new-fact accuracy, and MMLU accuracy -- but steering at a higher coefficient (1.25) suppresses hallucination further, down to the base model's baseline level, while prompting has no comparable way to be intensified (persona-vectors, §"J.7.2 Preventative prompting on a fact-acquisition task", p. 49). This exposes a genuine control-range gap between the two operationalizations of the same idea: a scalar coefficient is continuously tunable, while a system prompt's presence is close to binary; its wording can vary, but there is no well-defined operation that means "more" of it the way scaling $\alpha$ does for steering.
What it repurposes
Everywhere else in the paper, System-prompting is strictly an inference-time lever: it elicits trait expression to validate projection-based monitoring, and it is applied to an already-finetuned model as a mitigation baseline, prepending a negative instruction after the fact (persona-vectors, §"J.2 Comparison with system prompt-based mitigation", p. 45). Preventative prompting relocates the identical mechanism, conditioning a response via prepended instruction text, inside the training loop itself, which changes what the prompt is doing causally: at inference it competes against already-fixed weights, trying to overpower behavior already encoded, while during training it is present while the weights are still being formed, changing what the optimization needs to internalize in the first place. The result demonstrates that the "vaccine" idea behind preventative steering does not strictly require touching activations at all -- text conditioning during training reproduces much of the same protective effect.