Superposition — manifests as → Polysemanticity

A vector space contains only as many orthogonal directions as it has dimensions, so if a network genuinely represents more features than it has neurons, those extra features must share non-orthogonal directions rather than each owning a clean coordinate. A neuron's activation is a readout along one such shared, non-orthogonal direction, so when several crowded features write to it, the neuron ends up firing for each of them and looks polysemantic from the outside. The paper is explicit that this trade only becomes worthwhile under a further condition: 'without high sparsity, interference between non-orthogonal features prevents any performance gain from superposition,' meaning sparse activation is the load-bearing assumption connecting the storage strategy to the resulting symptom. This is why the paper treats polysemanticity as evidence to be explained by an underlying representational choice, rather than as a fixed property of what a neuron is.