A decades-old, informal idea resurfaces as an unsolved precursor

Several of the corpus's concepts have an explicit ancestor from decades-earlier scholarship, whether an informal principle that only foreshadowed the problem or an already-formal model borrowed wholesale, long before a modern paper gave the problem its own technical treatment. Asimov's first law prefigures accidents in ML informally, a single monolithic injunction with no mechanism for an engineer to act on, which the modern paper immediately fractures into five attackable causes. The frame problem supplies the classical diagnosis for negative side effects, a 1969 knowledge-representation difficulty resurfacing, largely unaddressed, in reinforcement learning objective design. Formal verification supplies the rigor Asimov's first law lacked, showing what actually rigorous harm-avoidance looks like once a community commits to it, for cyber-physical systems if not yet for modern ML. And the delusion box gives a minimal formal model of environmental embedding, a 2011 result that predates Concrete Problems and strips the wireheading failure down to a stark, already-demonstrated case. A different kind of ancestor supplies the 2017 paper's reward-learning loss outright rather than merely anticipating it: the Bradley-Terry model, a 1952 statistical model for estimating score functions from pairwise comparisons, is adopted unmodified to supply the loss for the reward model, with no engineering needed beyond swapping in trajectory segments for whatever the original model ranked. The paper reaches for a second decades-old idea just to explain the first: the Elo score, chess's rating system, supplies the explanatory analogy for the Bradley-Terry model, licensing the reward network's output to be read as an interval-scaled rating rather than a bare classifier score, even though the paper never computes anything resembling an actual Elo rating. The newly-added interpretability paper contributes a lineage of its own, with ancestors ranging from thirty years old to eighteen months old. Sparse dictionary learning is operationalized as sparse autoencoders: Olshausen & Field's 1997 formulation, alternating optimization that re-solves a fresh sparse-coding problem for every data point, becomes a single learned encoder function that produces a sparse code for any activation vector in one forward pass, with the decoder's weight matrix playing the role of the classical dictionary. L4-norm maximization reformulates sparse dictionary learning, Qu et al.'s 2019 result that the same sparse-coding search can be recast as a continuous optimization problem, maximizing a norm closely related to kurtosis, that the paper invokes to explain two of its own empirical findings rather than merely to motivate its method. Two much more recent ancestors are extended or validated rather than merely reformulated. Yun-dictionary-learning-is-extended-by-sparse-autoencoders describes what the extension over Yun et al.'s 2021 precursor actually changes: a single sparse dictionary trained across every layer at once becomes a separate dictionary per layer (and per MLP sublayer), the finer granularity that makes this paper's layer-by-layer interpretability and dead-feature comparisons possible, on top of an autointerpretability, causal-localization, and monosemanticity evaluation apparatus Yun et al. never built. Sharkey-interim-report-is-validated-at-scale-by-sparse-autoencoders is the closest ancestor of all, an unpublished 2023 AlignmentForum post by one of this paper's own co-authors that first proposed training sparse autoencoders on language model activations and showed the idea worked in toy settings; this paper is explicitly framed, in its own introduction and related work, as scaling that interim result up to a real language model and adding the evaluation machinery the original report lacked. Where the 1952-and-earlier lineage borrows an already-formal statistical model wholesale, this lineage's ancestors run from an informal decades-old optimization principle through an already-technical but small-scale demonstration to a co-author's own unpublished forerunner, each supplying a different piece of what the finished method needed.