OpenWebText
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
A public web-text dataset used as the source of text fragments (the first 50,000 lines) on which dictionary feature activations are sampled for the autointerpretability protocol.
Judging whether an already-trained feature is interpretable on the same text it was trained on risks a quiet kind of cheating, where the evaluation ends up re-testing familiar ground rather than checking for anything general. This corpus exists to avoid exactly that: a public dataset built from upvoted web links, originally assembled to stand in for a well-known model's private training data, reused here purely to supply fresh text for scoring, never for training anything.
The page covers where this corpus came from, how an automated scoring pipeline samples fragments of it and measures a feature's activation on every token, and why keeping it separate from the corpus features are trained on is a basic precaution against the evaluation quietly grading itself.
The evaluation-side corpus, distinct from the Pile
OpenWebText belongs to open-models-as-interpretability-testbeds for a narrower reason than the Pile: it does not train anything in this paper. Its entire role is to supply the text the autointerpretability protocol uses to judge, after the fact, whether an already-trained dictionary feature is interpretable -- keeping the judged text separate from the training text the dictionary was fit on.
A public stand-in for GPT-2's private training data
OpenWebText is an open-source dataset, assembled by scraping and filtering URLs shared on Reddit that received sufficient community upvotes, built specifically to reconstruct something close to OpenAI's WebText corpus, the private data GPT-2 was trained on but never released. It became a standard, widely reused public corpus in language-model research generally, including in Bills et al. (2023)'s original OpenAI neuron-explainer protocol for automatically interpreting individual units in a network -- the exact protocol this paper adopts and scales up, keeping the same source corpus that protocol was originally built around.
How the protocol samples from it
The autointerpretation protocol's first step takes the first 50,000 lines of OpenWebText, extracts a 64-token sentence fragment from each, and measures the target feature's activation on every token of that fragment, rescaling activations to integers between 0 and 10 (sparse-autoencoders, §"A Autointerpretation protocol", p. 11). Those rescaled fragments are what later steps of the protocol pass to GPT-4 for explanation-writing and to GPT-3.5 for activation simulation, so OpenWebText's role is entirely upstream: it decides which text the rest of the interpretability pipeline gets to see a feature activate on.