CAA vs. finetuning computational cost comparison — avoids the memory cost of → Supervised fine-tuning (SFT)

The resource gap follows directly from what each method computes. Generating a CAA steering vector needs only forward passes -- run the model on each contrastive pair, cache the residual-stream activations, average the differences -- so it fits on a single NVIDIA L40 GPU in under five minutes. Finetuning Llama 2 7B Chat on the same 1,000-example dataset needs gradients and optimizer state for every parameter, so the paper runs it across two L40 GPUs with data parallelism and the ZeRO optimizer (Rajbhandari et al., 2019) for memory partitioning, taking about ten minutes (contrastive-activation-addition, §"J Computational resources", p. 15). The extra GPU and the optimizer-sharding machinery are not incidental engineering choices; they are the price of backpropagation, paid because finetuning has to store and update every weight rather than compute one averaged direction. That asymmetry is why CAA can be re-run per behavior, per layer, almost as a diagnostic query, in a way finetuning a full model realistically cannot.