CAA vs. finetuning computational cost comparison
Steering Llama 2 via Contrastive Activation Addition — introduced
A resource-cost comparison finding that generating a CAA steering vector requires only forward passes and takes under 5 minutes on a single GPU, versus about 10 minutes on 2 GPUs to finetune Llama 2 7B Chat on 1000 examples.
Techniques for shaping model behavior often carry a hidden cost in GPU-hours, which matters when the real choice is between a method that works and one that's actually affordable to run. CAA's cost turns out to be small: producing a steering vector needs nothing but forward passes, no gradients, no backward pass at all.
The numbers sit side by side ahead: under five minutes on a single GPU to generate a CAA vector, against roughly ten minutes on two GPUs to finetune the same model on a thousand examples, a gap worth knowing before deciding which intervention to try first.
Under five minutes, one GPU, no gradients
This is a direct cost comparison the paper runs between generating a CAA steering vector and finetuning the same model on the same data. Generating a vector needs only forward passes — run the model on each contrast pair, cache residual-stream activations, average the differences — so it fits on a single NVIDIA L40 GPU in under five minutes per behavior (contrastive-activation-addition, §"J Computational resources", p. 15). Supervised fine-tuning (SFT) on the same 1,000-example dataset needs gradients and optimizer state for every model parameter; the paper runs it across two L40 GPUs with data parallelism and the ZeRO optimizer for memory partitioning (Rajbhandari et al., 2019), taking about ten minutes (contrastive-activation-addition, §"6 Comparison to finetuning", p. 6).
What the extra GPU and ten minutes are paying for
The gap is not incidental engineering overhead — it is the price of backpropagation. Finetuning has to store and update every weight in the model, while CAA computes one averaged direction and never touches a gradient. That asymmetry is offered within CAA combined with supervised finetuning comparison as part of why steering is attractive as a lever in its own right: a Steering vector can be regenerated per behavior, per layer, almost as a diagnostic query, in a way a full finetuning run realistically cannot. It is one leg of the paper's three-way comparison among System-prompting, Supervised fine-tuning (SFT), and Contrastive Activation Addition (CAA) as levers on the same underlying model.
Why the paper could afford to search so broadly
This gap in cost is not just an abstract selling point; it shapes what the paper itself was able to do experimentally. Multiple-choice behavioral evaluation and layer sweep sweeps a vector at every layer for every one of seven behaviors before any multiplier tuning even begins, and the open-ended evaluations then sweep multiple multiplier values on top of that. A search with that many configurations would be close to prohibitive if each point required a separate finetuning run at the cost documented in Appendix J; because a CAA vector costs only forward passes, sweeping layers and multipliers this broadly across seven behaviors on two model sizes stayed within a computational budget of forward passes and averaging, rather than dozens of independent gradient-based training runs.