Outlier dimensions — partially restores → Privileged basis

Despite the residual stream lacking a privileged basis in general, the paper cites a documented exception: 'a tendency for transformers to store information in the residual stream basis' (Dettmers et al., 2022), attributed to the Adam optimiser saving gradients with finite precision in that basis (Elhage et al., 2023). The paper's own feature search makes this concrete rather than just citational: searching the residual stream's default-basis coordinates for the dimensions most associated with the apostrophe feature, the top two most positive dimensions were both outlier dimensions, and one of the top two most negative dimensions was also an outlier dimension. So among the handful of coordinates that do carry a privileged, information-bearing role in this basis, outlier dimensions are disproportionately represented -- a partial, localized exception rather than the residual stream regaining a privileged basis wholesale.