Dictionary expansion factor (R) — governs emergence of → Dead features

Appendix E's sweep (Figure 16) shows that for residual-stream dictionaries, raising R keeps essentially the whole dictionary alive up to roughly 4x overcompleteness, with dead features only beginning to appear past that point -- so within that range R purely trades dictionary size against interpretability, with no hidden capacity cost. Push past that range, or move to MLP-sublayer activations, and the assumption breaks: MLP dictionaries develop large numbers of dead features even at alpha=0, with no sparsity pressure at all, showing the failure mode there is driven by the ReLU nonlinearity and MLP activation geometry rather than by R or the sparsity penalty. This is part of why the paper settles on different working hyperparameters for the two settings (alpha=8.6e-4 for the residual stream, alpha=3.2e-4 for MLPs) -- R cannot be tuned in isolation from where in the model the dictionary is trained.