Sparsity loss (L1 penalty on feature activations) — forces unit norm on → Dictionary feature

The alpha||c||_1 penalty only sparsifies meaningfully if dictionary features are held to a fixed scale, because c and M trade off against each other: doubling a feature's row norm in M lets the optimizer halve the corresponding coefficient in c while reconstructing the same activation, silently shrinking the sparsity loss without making the reconstruction any sparser in any real sense. The paper closes this loophole by normalizing each row of M -- each dictionary feature -- to unit norm, so the L1 penalty cannot be gamed by rescaling the dictionary instead of genuinely zeroing out coefficients (footnote 3). This constraint is what makes a 'dictionary feature' well-defined as a direction rather than a direction-with-arbitrary-magnitude: its only free parameter is which way it points, not how long it is. The same normalization is cited, via Sharkey et al. (2023) and Wright & Ma (2022), as part of why l1-penalized training can recover the actual ground-truth directions that generated the data, rather than some arbitrarily rescaled proxy for them.