Real-world chat data validation of persona vectors — validates against → LMSYS-Chat-1M

LMSYS-Chat-1M is the first of four real-world datasets tested precisely because it is 'not carefully filtered or curated' and contains 'a large fraction of toxic content' (persona-vectors, §"L Inducing persona shifts from real-world datasets", p. 55) - a deliberately messy, close-to-worst-case corpus rather than a clean benchmark. The validation protocol ranks its conversations by projection difference per trait and finetunes separately on the top-500 (high), bottom-500 (low), and 500 random samples, finding a consistent ordering: high-projection samples induce the strongest trait expression, low-projection samples the weakest (§"6.3 Validation on real-world chat datasets", p. 11). The result survives even after GPT-4.1-mini strips out samples that overtly display the trait, and the examples that remain in the filtered high-projection subset are ones a text-only judge would plausibly pass - underspecified continuations like 'keep writing the last story' for hallucination, where nothing in the prompt itself signals risk. Appendix L's comparison across the other three datasets (TULU3, ULTRACHAT200K, WildChat) then shows the effect weakens as curation increases, which is exactly why LMSYS-Chat-1M, as the least curated of the four, produces the sharpest demonstration: the method's value is clearest on data nobody has already cleaned, which is also the realistic condition practitioners scraping real conversations are most likely to face.