L4-norm maximization (sparse dictionary search reformulation) — reformulates → Sparse dictionary learning / sparse coding

Qu et al. (2019) show that the search for a sparse, overcomplete dictionary can be recast as a continuous optimization problem: searching for unit-norm directions that maximize their l4 norm over the data. This reformulation works because a direction that recovers one of the true sparse-coding basis vectors will have activations that are mostly near zero with occasional large spikes -- exactly the heavy-tailed profile the l4 norm, unlike the l2 norm, rewards, so maximizing it is a smooth proxy for finding maximally sparse-activating directions. This paper invokes the result to explain two of its own findings: why the correlation between a dictionary feature's autointerpretability score and the skew/kurtosis of its activations is positive, and why Independent Component Analysis, which directly maximizes non-Gaussianity, is consistently the strongest non-dictionary-learning baseline it tests against.