Sample-level projection-based data filtering
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — introduced
An extension of projection difference from the dataset level to individual training samples, used to flag which specific examples within a dataset are likely to induce a persona shift. Individual samples from trait-inducing datasets are highly separable from control samples by their projection values, and combining projection-difference filtering with LLM-judge filtering yields more robust filtering than either alone.
Knowing that a dataset is risky on average doesn't tell you which specific examples inside it are actually the problem — a dataset-level warning isn't yet a tool you can use to clean anything. This method takes the same underlying signal and applies it row by row instead, turning a corpus-wide diagnostic into an example-by-example filter.
Computing the per-example score without any averaging is the mechanical core, alongside how its cutoff gets calibrated against a dataset believed to be clean rather than fixed by hand. Set against a more familiar approach — an LLM reading each response directly — the two turn out to catch different problems and work best combined rather than as substitutes.
From a dataset average to a per-row decision
Projection difference as originally defined answers a question about an entire training corpus: averaged over every example, will this dataset shift the model's persona? Sample-level filtering strips out the averaging and applies the identical per-example term — the difference in projection between a training response and the base model's own response to the same prompt, projected onto the unit-normalized persona vector — to each row individually, turning a corpus-level diagnostic into an actual data-cleaning tool that flags which specific examples to remove (persona-vectors, §"6.1 Predicting post-finetuning behaviors from data", p. 9; §"6.2 Sample-level detection of problematic data", p. 10). Histograms of the resulting per-sample projections show this separation is often clean: individual samples from trait-inducing datasets are largely separable from matched control samples by projection value alone, and this holds not only for datasets intentionally designed to elicit a trait but also for "EM-like" datasets whose flaws are domain-specific rather than trait-explicit — training on flawed math reasoning, for example, still separates cleanly on the evil projection axis even though the dataset never mentions evil (persona-vectors, §"6.2 Sample-level detection of problematic data", p. 10).
Calibrating a threshold without ground truth
Turning a per-sample projection value into a keep/discard decision requires a threshold, and the paper calibrates one empirically rather than fixing an arbitrary cutoff: each trait's filtering threshold is set at the 95th percentile of that metric's own distribution over 20,000 samples from ULTRACHAT200K, a dataset chosen because finetuning on it does not lead to noticeable persona shifts (persona-vectors, §"K Sample-wise filtering", p. 53). On a mixed Opinion Normal + Opinion Mistake II dataset used as the test case, this yields quite different-looking thresholds across metrics and traits — a projection-difference cutoff of 4.32 for evil, 4.98 for sycophancy, and 2.25 for hallucination, against LLM-judge-score cutoffs of effectively 0, 31.74, and 72.71 respectively (persona-vectors, §"K Sample-wise filtering", p. 53). Defining "clean" relative to an observed reference distribution, rather than a fixed number chosen in advance, is what makes the same procedure reusable across traits whose score distributions otherwise look nothing alike.
Complementary to, not a replacement for, an LLM judge
Compared directly against GPT-4.1-mini-based filtering on the same mixed dataset, the two approaches do not simply agree with one being a subset of the other: LLM-based filtering performs best on hallucination, projection-difference-based filtering is most effective for sycophancy, both methods are effective on evil and completely eliminate trait shift there, and the combined filtering strategy consistently yields the best results across all three traits (persona-vectors, §"K Sample-wise filtering", p. 54). The likely explanation is that the two filters see different things: sycophantic and evil content are often stylistically legible in the response text an LLM judge reads, while what makes a sample shift a model's persona is not always what makes it read as displaying the trait in the first place — a distinction visible to an activation-based signal that a purely semantic judge can miss, and vice versa for hallucination.