Inference-Time Intervention (ITI)
Steering Llama 2 via Contrastive Activation Addition — inherited
A prior technique (Li et al., 2023) using linear probes to identify a sparse set of "truthful" attention heads from a contrastive question-answering dataset, then shifting activations along the true/false Mean Difference vector during inference to improve truthfulness.
Not every activation-steering technique reaches into the same part of a transformer. Inference-Time Intervention takes a narrower path: first use linear probes to hunt down a small set of attention heads that seem to track truthfulness, then nudge only those heads' activations along a Mean Difference direction during generation.
ITI shares the same underlying Mean Difference math as CAA, but applies it to a hand-selected sparse set of attention heads rather than the whole residual stream, and was built specifically around truthfulness rather than a broader menu of behaviors, a contrast CAA draws out explicitly when placing itself among prior techniques.
Truthful heads, found by probing
Li et al. (2023) introduce Inference-Time Intervention (ITI) to make models more truthful without retraining them. The method first trains linear probes on a contrastive question-answering dataset to predict, from a model's internal activations, whether it is about to answer truthfully or not; from these probes it selects a sparse set of attention heads whose activations carry the strongest truthful/untruthful signal. At inference time, ITI shifts only those heads' activations along the vector connecting the mean of the "true" activations to the mean of the "false" activations -- the same Mean Difference construction CAA also uses. The result improves performance on adversarial benchmarks like TruthfulQA while barely affecting fluency, and needs comparatively little data to work (contrastive-activation-addition, §"2 Related work", p. 2).
Same extraction, no head search
CAA is explicit that the two methods "employ[s] the same Mean Difference vector extraction approach as CAA" (contrastive-activation-addition, §"2 Related work", p. 2), which means the entire difference between them is about where the resulting direction gets applied, not how it is computed. ITI needs its probe-and-search step because it only intervenes at a sparse set of attention heads; CAA skips that step because "our technique can be applied directly to the residual stream without searching for individual attention heads" (contrastive-activation-addition, §"2 Related work", p. 2). That simplification is paired with broader scope: "we validate the approach on a broader range of alignment-relevant behaviors in models trained using RLHF" (contrastive-activation-addition, §"2 Related work", p. 2) -- seven behaviors on Llama 2 7B and 13B Chat, rather than one truthfulness axis on a base model.
Shared machinery, different granularity
Because ITI and CAA both build their direction as a difference of class means in the Residual stream -- or, for ITI, at particular attention heads within it -- they can be understood as two settings of the same extraction procedure applied at different granularities: a handful of probed heads for ITI, the full stream for CAA. That shared foundation is what lets CAA's related-work comparison stay narrow: it does not need to argue that its extraction method is better than ITI's, only that removing the head-search step generalizes the same underlying Mean Difference (MD) vector extraction to more behaviors and to larger, RLHF-trained models without losing effectiveness.