Interpretability-vs-kurtosis/skew correlation analysis — explains variation in → Autointerpretability score

Appendix C.4's Table 2 breaks the correlation between interpretability score and a feature's activation statistics down by moment, and the pattern is not uniform: mean correlates at -0.09 and variance at 0.02, both essentially nothing, while skew correlates at 0.20 and kurtosis at 0.15 (the main text quotes slightly different figures for the same analysis, 0.19 and 0.24 respectively). That split matters for what it implies the score is picking up on: it is not rewarding features that simply activate more strongly or more variably, only ones whose activation distribution has a heavier tail relative to a Gaussian, i.e. large but rare spikes rather than broad or high-magnitude activation generally. The paper's proposed mechanism is that interference from other features sharing a direction in superposition should be roughly Gaussian by the central limit theorem, so a feature standing out from that Gaussian background is more likely to be a real, cleanly-recovered feature rather than noise. This is also offered as the explanation for ICA's baseline performance: since ICA explicitly searches for non-Gaussian, heavy-tailed directions, it scores closer to the SAE's dictionary features than PCA or the default basis do.