Automated Circuit Discovery (ACDC)
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
A prior algorithm (Conmy et al., 2023) for automatically discovering the subgraph of a network responsible for a behavior. This paper reuses ACDC's ordering procedure (its Algorithm 4.1) to rank dictionary features by their individual causal contribution to reducing KL divergence on the IOI task, producing the feature subset used in the dictionary-feature-patching experiments.
An algorithm built to automate a job that used to take researchers weeks of manual, head-by-head detective work turns out to do real work twice over in a later paper — once with a citation attached, and once, quietly, as the same underlying logic reused without being named again. Both times, the point is the same: causal importance should be established by deliberately removing something and measuring the damage, not just by eyeballing what looks interpretable.
The page covers what the algorithm does in general, the specific step it supplies for ranking which learned features to intervene on first, and its unremarked second appearance later in the same paper, powering a different method under a different name.
A borrowed algorithm doing two jobs in this paper
Automated Circuit Discovery is inherited machinery, not something this paper invents, but it does real work twice over: once explicitly, supplying the feature-ranking step inside Dictionary-feature patching procedure, and once implicitly and uncredited, in the logic underneath Automatic feature circuit detection. Both uses sit inside the causal-proof-by-patching theme's core claim that causal importance should be established by intervention, not by inspection alone.
What ACDC does, in general
ACDC (Conmy, Mavor-Parker, Lynch, Heimersheim and Garriga-Alonso, 2023) automates a task previously done by hand, as in Wang et al. (2022)'s manual reverse-engineering of the IOI circuit: given a neural network represented as a computational graph of components (attention heads, MLP sublayers, embeddings) connected by edges, it searches for the minimal subgraph that preserves performance on a chosen metric, iteratively ablating components and measuring how much each removal degrades that metric, so that what remains after pruning is offered as the circuit responsible for a behavior.
What this paper borrows, specifically: Algorithm 4.1
Section 4.1 imports ACDC's Algorithm 4.1 directly, but repurposes it for ranking rather than pruning down to a strict subgraph: it treats every dictionary feature as an independent node in a flat computational graph, in which each feature contributes an independent change to the target metric, and orders features by how much patching each one individually reduces the KL divergence from a target IOI output, averaged over 50 test examples (sparse-autoencoders, §"4.1 Adapting activation patching to dictionary features", p. 6). The resulting ordering, rather than a fixed circuit, is what supplies the feature subsets used throughout the patching experiments, applied separately to the dictionary-feature and PCA decompositions alike.
An unremarked second life in Section 5.3
The same ablate-and-rank logic resurfaces one section later, in the paper's circuit-detection method for individual features, without ACDC or Conmy et al. being cited again there (sparse-autoencoders, §"5.3 Intermediate features: dictionary features allow automatic circuit detection", p. 7). There, instead of ranking features by their effect on a whole task's KL divergence, the same ablate-measure-rank-recurse pattern ranks previous-layer features by their effect on a single target feature's own activation strength. The paper presents Section 5.3's method as a fresh, self-contained procedure, but the underlying move -- ablate a candidate cause, measure the drop in a target metric, order, and optionally recurse -- is the one Algorithm 4.1 supplied a section earlier.