Reconstruction-substitution perplexity metric — independently corroborates → Reconstruction loss

Reconstruction loss is measured in squared-error units on cached activations (Equation 4), a number with no inherent interpretation of how much it actually matters to the model's behavior. The reconstruction-perplexity metric translates that abstract gap into a concrete, task-relevant cost by substituting the SAE's reconstruction directly into the model's forward pass and re-measuring next-token prediction quality: for Pythia-70M's layer-2 residual stream, doing so raises perplexity on the Pile from 25 to 40. That roughly 60% jump in perplexity from a single-layer substitution is the paper's evidence that reconstruction loss, though small-looking in absolute squared-error terms, is 'not yet near zero' in any sense that matters -- the dictionary is measurably failing to capture information the model actually uses. The two metrics are kept distinct in the paper: reconstruction loss is the differentiable quantity optimized during training, while the perplexity metric is a non-differentiable, downstream check computed only after training to validate what the loss curve alone cannot show.