Linear representation hypothesis
Steering Llama 2 via Contrastive Activation Addition — inherited; Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
A formalization (Park et al., 2023) of the idea that high-level concepts are encoded as linear directions in an LLM's activation space, cited by CAA as part of the broader motivation for why activation-engineering techniques like steering vectors work.
Subtracting one set of activations from another and calling the result a "behavior direction" only makes sense if behaviors are actually stored as directions in the first place, rather than some more tangled shape that a simple average would just blur together. That assumption has a name.
It's called the linear representation hypothesis: high-level concepts sit along linear directions in a model's activation space, part of the theoretical backdrop CAA leans on to justify why a technique as simple as Mean Difference should work at all.
The hypothesis, stated precisely
Park et al. (2023) formalize an idea that had been implicit in activation-engineering work for years: that high-level, human-interpretable concepts are encoded as linear directions in an LLM's activation space (contrastive-activation-addition, §"2 Related work", p. 3). Concretely, for a concept with a positive and negative pole -- refusing versus complying, truthful versus false, sycophantic versus not -- the hypothesis says there is a direction $\theta_c$ in activation space such that a context's activation approximately decomposes as $$a(x) \approx \alpha(x)\,\theta_c + a_\perp(x),$$ where $\alpha(x)$ is a scalar measuring how strongly concept $c$ is expressed in $x$, and $a_\perp(x)$ is everything else about $x$'s representation, roughly independent of $\theta_c$. Two consequences follow directly: $\alpha(x)$ can be read out by projecting $a(x)$ onto $\theta_c$, and $\alpha(x)$ can be changed by moving $a(x)$ along $\theta_c$ -- exactly what adding a Steering vector does.
Licensing the vector arithmetic of steering
This is why the hypothesis matters for CAA specifically rather than as generic background: it is what makes it coherent to expect that adding one vector to activations should shift a whole behavior rather than producing noise. If concepts really are linear directions, then a behavior like sycophancy or refusal has, in principle, a direction, and pushing a model's activations along it -- what CAA steering vector construction (Mean Difference over multiple-choice contrast pairs) does with a chosen multiplier -- should move the output distribution toward or away from that behavior in a controllable, roughly dose-dependent way. CAA does not derive the hypothesis; it inherits it as motivation, stated directly in the related-work discussion, and then hands evidence back to it.
Practice first, theory after
The paper is careful about the historical order here: "beyond steering behaviors, work on activation engineering has also motivated a formalization of 'linear representation' (Park et al., 2023)" (contrastive-activation-addition, §"2 Related work", p. 3) -- naming the hypothesis as a response to the accumulating empirical successes of Activation Addition (ActAdd), Inference-Time Intervention (ITI), Representation Engineering (Zou et al. 2023), and In-Context Vectors (ICV), not the theory those methods were built to test. CAA continues the pattern one step further, offering two of its own findings as further support: Behavioral clustering shows a two-dimensional linear projection of activations cleanly separating by behavior, and Token-level cosine similarity analysis shows steering vectors acting as feature detectors -- both consistent with a single linear direction carrying the behavior, cited as evidence for the hypothesis rather than as independent inventions.