LMSYS-Chat-1M
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
A real-world dataset (Zheng et al., 2024) of one million conversations between users and 25 different LLMs, largely uncurated and containing substantial toxic content. Used both as the primary real-world validation source for projection-difference-based data screening and as chat-domain training data for the sparse autoencoders used to decompose persona vectors.
Every other training dataset used elsewhere in this paper was built on purpose, engineered to produce a specific trait at a chosen intensity. This one is the opposite: a million real conversations collected without much filtering at all, kept deliberately messy because that messiness is exactly what makes it a meaningful test.
Its contents, and why it was chosen specifically for a lack of curation, open the account. A central role validating that a purely pre-training signal can predict which real conversations will shift a model's character after finetuning, and a smaller second role supplying ordinary chat data for an unrelated interpretability method, round things out.
What LMSYS-Chat-1M is
LMSYS-Chat-1M is a prior real-world dataset (Zheng et al., 2024) of one million conversations collected from users interacting with 25 different large language models through a public chat interface, released to support research on how models are actually used and where they fail outside curated benchmarks. It is deliberately not filtered for quality or safety at collection time, so it captures the full range of real user behavior, including large amounts of low-effort, adversarial, and toxic content alongside ordinary conversation. The persona-vectors paper describes it in exactly these terms: "this dataset is not carefully filtered or curated, and thus contains a large fraction of toxic content" (persona-vectors, §"L Inducing persona shifts from real-world datasets", p. 55) — the least curated of the four real-world datasets the paper tests against.
Primary role: validating projection-based data screening on messy real-world data
LMSYS-Chat-1M is the paper's primary real-world test of Real-world chat data validation of persona vectors: each conversation's Projection difference (pre-finetuning data-screening metric) is computed per trait, and models are separately finetuned on the top-500, bottom-500, and 500 random samples. The result is a consistent ordering — high-projection samples induce the strongest trait expression, low-projection samples the weakest (persona-vectors, §"6.3 Validation on real-world chat datasets", p. 11) — that survives even after GPT-4.1-mini strips out any sample that overtly displays the target trait. What remains in the filtered high-projection subset can be genuinely non-obvious: for hallucination, the method surfaces underspecified continuations such as "Keep writing the last story," where nothing in the prompt itself signals risk of fabrication (persona-vectors, §"6.3 Validation on real-world chat datasets", p. 11). Because it is the least curated of the four real-world datasets tested (the others being TULU3, UltraChat200K, and WildChat), the effect is also sharpest here — the paper notes results "tend to be weaker for more refined datasets, which don't have clear examples of bad behavior ... in the first place" (persona-vectors, §"L Inducing persona shifts from real-world datasets", p. 55).
Secondary role: training data for the SAE decomposition
LMSYS-Chat-1M also supplies the chat-domain component of the training mix for the Sparse autoencoders (SAEs) later used in SAE decomposition of persona vectors, alongside pretraining text (The Pile) and a small amount of misalignment data — a choice motivated by the hypothesis that the features underlying Emergent misalignment "would be most prevalent in chat data" (persona-vectors, §"M.1 SAE training details", p. 59). The dataset is used a second time within that same appendix as a source of max-activating examples: to interpret what an individual SAE feature detects, the paper runs the model over 500,000 samples from LMSYS-Chat-1M and The Pile and inspects which samples activate that feature most strongly (persona-vectors, §"M.3.1 Max activating examples", p. 60).