Principal Component Analysis (PCA)
Steering Llama 2 via Contrastive Activation Addition — inherited; Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
A linear dimensionality-reduction technique that projects data onto axes of maximal variance. CAA uses it (via scikit-learn) to visualize whether contrastive-dataset activations linearly separate by target behavior.
Reducing a pile of high-dimensional activations down to the handful of directions that capture the most variance in the data is a decades-old, general-purpose trick, and it doesn't care what kind of data you feed it or what you're trying to learn from the result. One paper points it at a contrastive dataset as a fast visual check for whether behavior-relevant activations separate cleanly; a later paper repurposes the same technique as a rival decomposition, competing head-to-head against learned dictionary features on the same interpretability and causal-precision tests.
This page follows Principal Component Analysis across both roles: the diagnostic plot that helps confirm a steering vector is measuring something real, and the full competing baseline that a later paper's dictionary features are measured against and consistently outperform.
Projecting onto directions of maximal variance
Principal Component Analysis (PCA) is a general-purpose, unsupervised technique for reducing the dimensionality of data while preserving as much of its variation as possible. Given a set of points -- here, residual-stream activation vectors -- centered so their mean is zero, PCA looks for the direction along which the data varies the most, then the next such direction orthogonal to the first, and so on. Formally, for centered data $X$ with covariance matrix $\Sigma$, the first principal component is $$v_1 = \arg\max_{\|v\|=1} \mathrm{Var}(v^\top X) = \arg\max_{\|v\|=1} v^\top \Sigma v,$$ and subsequent components maximize the same quantity subject to being orthogonal to all previous ones; equivalently, the components are the eigenvectors of $\Sigma$, ordered by decreasing eigenvalue. Projecting data onto the top two or three components gives the best low-dimensional linear summary of the data's spread, which is what makes it possible to plot high-dimensional activations on a page and look for structure by eye.
CAA's use: a linear-separability check
CAA quotes its own working definition directly: "Principal Component Analysis (PCA) is a linear dimensionality reduction technique. It linearly projects the data onto a new coordinate system, where the axes (principal components) are selected to account for the most significant variance in the data" (contrastive-activation-addition, §"3.2 Visualizing activations for contrastive dataset analysis", p. 4). The paper applies it, via the Scikit-learn package, directly to the problem PCA visualization of contrastive activations is built to solve: projecting contrastive-dataset activations down to their top two components and checking by eye whether the resulting scatter separates by behavior. No new projection method is introduced -- the contribution is the specific diagnostic use of an existing one.
Unsupervised agreement as evidence
What makes PCA more than a plotting convenience here is that it uses no behavior labels at all, unlike the supervised Mean Difference (MD) vector extraction CAA actually extracts steering vectors with. That two such differently-informed procedures land on similar directions -- the finding CAA cites from Linear representations of sentiment for the specific case of sentiment -- is part of the broader case for the Linear representation hypothesis: if a behavior's representation were not close to linear, an unsupervised, variance-maximizing, two-dimensional projection would have no particular reason to align with a supervised difference-of-means direction, let alone to reveal any separation at all.
A second career: baseline for a later paper's dictionary features
A different paper later repurposes PCA in a role that has nothing to do with visualizing behavioral separation: as one of four fixed baselines -- alongside the default basis, random directions, and ICA -- that sparse dictionary features are measured against. Here PCA is not a diagnostic plot but a full competing decomposition of a language model's activations, estimated online over the same 5-50M activation vectors used to train the autoencoders themselves (sparse-autoencoders, §"3.2 SPARSE DICTIONARY FEATURES ARE MORE INTERPRETABLE THAN BASELINES", p. 3). It is compared on two separate grounds: the Autointerpretability score of its components, and, on the Indirect Object Identification task, how few of its components need to be patched into the residual stream, and by how little, to reach a given KL divergence from the target output -- a comparison PCA loses on both counts (sparse-autoencoders, §"4.2 PRECISE LOCALISATION OF IOI DICTIONARY FEATURES", p. 6). Tellingly, the gap is not just about PCA's fixed, variance-ranked axes being unable to rotate to fit an arbitrary target: patching an equal-size set of features from the same autoencoder architecture trained with no sparsity penalty at all ($\alpha=0$) also underperforms the sparse dictionary and looks more like PCA, indicating that what actually produces cleaner causal localization is the sparsity penalty forcing few features to matter per token, not just the extra flexibility of learning a dictionary at all (sparse-autoencoders, §"4 IDENTIFYING CAUSALLY-IMPORTANT DICTIONARY FEATURES FOR INDIRECT OBJECT IDENTIFICATION", p. 5). PCA's variance-maximizing objective, unchanged from its general definition above, simply optimizes for the wrong thing here: it spreads a corpus's signal across whichever directions carry the most variance overall, which need not be the few directions that causally drive one specific behavior like IOI.