The Pile
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
A large public webtext corpus (Gao et al., 2020) used to generate the activation datasets the sparse autoencoders are trained on, and to measure perplexity when testing how much information is lost by reconstructing a layer's activations from its dictionary features.
Every claim this paper makes about interpretability, causal precision, or how cleanly a feature's meaning holds together ultimately traces back to the same pile of raw text: the corpus a small open language model was pretrained on, later reused to generate the very activations that get decomposed into learned features.
The page covers what this corpus is and why it was built, the two separate jobs it does here — supplying the raw material for feature training and setting the yardstick for a reconstruction-fidelity check — and why its role is kept deliberately distinct from a second corpus used only for evaluating whether the resulting features are interpretable, so training data and judging data never quietly overlap.
The corpus underneath every activation dataset in the paper
The Pile belongs to open-models-as-interpretability-testbeds for a simple reason: every sparse autoencoder in this paper is trained on activations sampled from Pythia models running over the Pile, so the Pile is the raw material every downstream claim about interpretability, causal precision, and monosemanticity is ultimately built from.
An 800GB corpus built for training large language models
The Pile (Gao, Biderman, Black, Golding, Hoppe, Foster, Phang, He, Thite, Nabeshima et al., 2020) is a roughly 800GB, openly released corpus assembled from 22 diverse sources, including academic text, web crawls, books, and code, designed specifically to serve as pretraining data for large language models where existing public corpora were too narrow or too small. It is also, notably, the corpus Pythia itself was pretrained on, so using it here to generate activation datasets keeps the sparse autoencoders' training distribution matched to the language model they are interpreting.
How this paper uses it: two separate jobs
The Pile does two distinct jobs in this paper. First, it supplies the raw text used to build every activation dataset: the model under study is run over Pile text while caching activations at a chosen layer, producing 5 to 50 million activation vectors used to train each autoencoder over 1 to 3 epochs (sparse-autoencoders, §"B Sparse autoencoder training and hyperparameter selection", p. 12); the dictionaries used for the IOI-patching results specifically are trained on just the first 10,000 elements of the Pile, about 7 million activations, on layer 11 of Pythia-410M's residual stream (sparse-autoencoders, §"4 Identifying causally-important dictionary features for indirect object identification", p. 5). Second, it supplies the yardstick for a fidelity check: substituting Pythia-70M's layer-2 residual stream activations with their sparse-autoencoder reconstruction raises the model's perplexity on the Pile from 25 to 40 (sparse-autoencoders, §"6.2 Limitations and future work", p. 9).
Training corpus, not evaluation corpus
The Pile's role here is deliberately different from OpenWebText's: the Pile generates the activations the autoencoders are trained and evaluated for reconstruction fidelity on, while OpenWebText is reserved for a separate downstream purpose, sourcing the text fragments the autointerpretability protocol samples feature activations from. Keeping the two corpora separate means the text used to build a feature dictionary is not identical to the text used to judge whether that dictionary's features are interpretable, a basic precaution against the evaluation simply re-testing the training distribution.