GPT-4.1-mini
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
OpenAI's smaller GPT-4.1 model variant, used throughout the paper as the automated trait-expression-score judge and, in the data-screening appendices, as an LLM-based data filter for identifying samples that overtly exhibit a target trait.
Nearly every measurement in this paper eventually traces back to one model's judgment about how strongly a response displays a given trait, which makes that judgment something worth scrutinizing rather than trusting on faith. This is the model doing that judging throughout.
Producing a continuous score from its own output logits, rather than a single sampled guess, is the core mechanism, checked directly against human raters before being trusted and reused for a separate coherence check. A second job surfaces elsewhere in the paper: filtering risky training data by reading response text directly, compared against a very different, activation-based way of catching the same problem.
The trait-expression-score judge
GPT-4.1-mini is the automated judge behind the paper's central measurement, the trait expression score: given a model transcript and a trait-specific evaluation rubric generated by Claude 3.7 Sonnet, it is prompted to output a number between 0 and 100 indicating how strongly the trait is expressed (persona-vectors, §"2.1 Generating trait-specific artifacts", p. 3). Rather than trusting whatever single token the model happens to sample, the paper retrieves the top-20 logits from its output distribution, isolates the subset corresponding to integers 0-100 (each a single token in OpenAI's tokenizer), and computes a logit-weighted sum over just those candidates to obtain a continuous score (persona-vectors, §"B.1 Evaluation scoring details", p. 28).
Validated against human judges, reused for coherence
Because nearly every result in the paper depends on this score, it is checked against human judgment directly: two of the paper's authors served as human raters over 300 pairwise comparisons of high- versus low-scored responses (50 pairs per trait, 2 raters each), agreeing with GPT-4.1-mini's ranking 97% of the time for evil, 92% for sycophancy, and 95% for hallucination — 94.7% overall (persona-vectors, §"B.2 Checking agreement between human judges and LLM judge", p. 28). GPT-4.1-mini also produces a separate coherence score used to check that steering interventions do not degrade general response quality, rating each response 0-100 by the same logit-weighted methodology (persona-vectors, §"5.1 Post-hoc steering mitigates behavioral shifts", p. 8).
A second role: data filter
In the data-screening appendices, GPT-4.1-mini is repurposed as an LLM-based filter for identifying training samples that overtly exhibit a target trait, compared directly against the paper's own activation-based projection-difference filtering. On a mixed dataset of Opinion Normal and Opinion Mistake II responses, each method calibrated to its own 95th-percentile threshold on a clean reference dataset, the two methods split unevenly by trait: GPT-4.1-mini alone performs best for hallucination, projection difference alone is strongest for sycophancy, and both independently eliminate trait shift for evil, with the combined filter giving the most consistent result across all three (persona-vectors, §"K Sample-wise filtering", p. 53-54). That GPT-4.1-mini is the stronger single filter for hallucination — not merely a text-reading complement to the activation-based signal — cuts against the simpler story that semantic judges and activation probes always catch different things.