Sparse autoencoders (SAEs)
Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
An unsupervised dictionary-learning technique (Cunningham et al., 2023; Bricken et al., 2023) that decomposes model activations into a larger set of sparse, individually interpretable feature directions. The paper trains BatchTopK SAEs (Bussmann et al., 2024) on Qwen-2.5-7B-Chat's residual stream, on a mix of pretraining, chat, and misalignment data, as a complementary, unsupervised approach to obtaining persona-relevant directions.
A model's raw internal activations rarely break cleanly along the lines of individual human concepts, since a network packs in far more distinct ideas than it has dimensions to represent them separately. Sparse autoencoders are a prior, unsupervised way of pulling interpretable structure back out of that crowded representation, without needing to know in advance what concepts you're looking for.
The general technique comes first, alongside the specific version trained for this paper on real chat and pretraining data. Why having one already trained beforehand matters comes last: it becomes a general-purpose tool that any later-extracted trait direction can be compared against, without repeating the paper's own supervised extraction process from scratch.
What sparse autoencoders are
Sparse autoencoders are a prior, unsupervised dictionary-learning technique for interpretability, predating this paper's own persona-vector extraction method. The core idea, developed by Cunningham et al. (2023) and Bricken et al. (2023) and cited by the paper as offering "a complementary approach to obtaining meaningful directions" (persona-vectors, §"8 Limitations", p. 13), addresses a specific problem: individual neurons in a network's activations typically do not correspond to single human-interpretable concepts, because models pack far more concepts into their representations than they have dimensions available (the "superposition" hypothesis). An SAE is trained to reconstruct a layer's activations while passing them through a much wider, sparsely-active hidden layer, forcing each token's activation to be explained by only a handful of active "features" out of a much larger dictionary, under the hypothesis that many of the resulting individual features will turn out to be interpretable, unlike the raw activation dimensions they were trained on. Unlike methods that require a target concept specified in advance, SAE training needs no labels at all: the dictionary of features is discovered purely from unlabeled activations.
The SAEs trained for this paper
The paper's own related-work section places SAEs among several prior methods for extracting interpretable directions: "another approach to obtaining interpretable directions is to train sparse autoencoders (SAEs), which find directions in an unsupervised way" (persona-vectors, §"7 Related work", p. 12). Concretely, it trains BatchTopK SAEs (Bussmann et al., 2024) on every fourth layer of Qwen-2.5-7B-Chat's residual stream, using a dictionary-learning codebase adapted from Marks et al. (2024), with 131,072 features against a residual-stream dimension of 3,584 — roughly a 37x expansion factor — and reports results at sparsity level k = 64 active features per token (persona-vectors, §"M.1 SAE training details", p. 59). The training mix combines pretraining text (The Pile), chat data (LMSYS-Chat-1M), and a small amount of misalignment data (Betley et al.'s insecure-code dataset and Chua et al.'s bad-medical-advice dataset), chosen because the authors were originally chasing Emergent misalignment, not persona vectors specifically (persona-vectors, §"M.1 SAE training details", p. 59).
A trait-agnostic complement to the paper's own extraction pipeline
Training even one of these SAEs is a far heavier investment than the paper's own extraction method: the Automated persona vector extraction pipeline needs only 400 rollouts per trait and a single mean-difference computation, while an SAE at this width requires the standard large-scale unsupervised dictionary-learning procedure, repeated separately for each residual-stream layer. The payoff is that the SAE is trained once and is trait-agnostic — any persona vector, for a trait targeted by the extraction pipeline or not, can later be projected onto it and decomposed via SAE decomposition of persona vectors, without rerunning contrastive extraction. The paper frames this as a way past its own supervision requirement: "SAEs may therefore enable unsupervised discovery of persona-relevant directions, including specific traits that cannot be easily elicited through prompting" (persona-vectors, §"8 Limitations", p. 13).