Mechanistic interpretability
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
A research agenda that seeks to understand neural networks by reverse-engineering their internal computations into human-understandable parts, aiming to enable verification, targeted editing, and mitigation of risks such as deceptive behavior. Named in the introduction as the field this paper's sparse-autoencoder method aims to advance, offering it as 'a foundation for future mechanistic interpretability work.'
A system can act competently while doing so for reasons nobody outside it can state in words, and that gap stops being a minor inconvenience once the system is capable enough that acting for the wrong reasons could matter, including, in the most unsettling version, deceiving the people watching it.
What follows traces the field built to close that gap by reverse-engineering a network's computation into pieces small enough to actually understand, the ambitious endpoint that motivates the effort, and why the field's original default unit for that decomposition, the individual neuron, turned out to stall.
The field this paper positions itself within
Mechanistic interpretability is the research agenda of understanding a neural network by reverse-engineering its internal computation into smaller, human-understandable parts rather than treating it as an opaque function. The name and program grew out of Anthropic's circuits-style analysis of vision models (Olah et al., 2020) and were extended to transformer language models (Elhage et al., 2021; Cammarata et al., 2021). This paper situates its contribution inside that agenda explicitly, opening by naming the field as a response to the risk that capable AI systems 'make decisions for reasons we do not understand,' including the possibility that they 'deceive humans in order to accomplish undesirable goals' (Ngo et al., 2022) (sparse-autoencoders, §"1 INTRODUCTION", p. 1), and closing by offering its method 'as a foundation for future mechanistic interpretability work' (sparse-autoencoders, §"ABSTRACT", p. 1).
The field's most ambitious version of success
What achieving mechanistic interpretability is meant to enable, according to the paper's own framing, is the ability to 'reverse engineer parts of [a network's] internal processes and make targeted changes to them' (sparse-autoencoders, §"1 INTRODUCTION", p. 1). Pushed to its logical completion, that ambition becomes Enumerative safety: a complete, human-understandable list of a model's features precise enough to guarantee it will not perform dangerous behaviours such as deception (sparse-autoencoders, §"6.3 CONCLUSION", p. 9). The paper frames its own contribution as a step toward that goal without claiming to reach it, since its dictionaries still leave reconstruction error, and therefore some of the model's computation, unexplained.
Why the neuron-level version of the program stalls
The field's basic method requires breaking a network down into units small enough to study one at a time; the paper notes that using individual neurons this way 'has had some success' (Olah et al., 2020; Bills et al., 2023) (sparse-autoencoders, §"1 INTRODUCTION", p. 1), which made neurons the default unit before this paper's method existed. Polysemanticity undercuts that default directly, since a neuron firing for several unrelated concepts resists being assigned a single human-readable role no matter how carefully it is studied. The paper's Sparse autoencoders (SAEs) are offered as a substitute unit of analysis, whose Dictionary features are shown across Sections 3 through 5 to be more interpretable, more causally precise, and closer to Monosemanticity than the neuron basis they aim to replace.