CAA as an adversarial red-teaming tool (proposed application)
Steering Llama 2 via Contrastive Activation Addition — introduced
A suggested future application using CAA as an adversarial intervention to trigger unwanted model behaviors more efficiently than prompting alone, on the premise that behaviors easily elicited via small internal perturbations may also be reachable in deployment, while inability to elicit them could indicate stronger safety.
Finetuning and RLHF can make a model much less likely to say something harmful, but "much less likely" isn't the same as "unable to," and jailbreaks keep surfacing the gap between the two. Systematically searching for the prompts that expose that gap by hand is slow and unreliable.
A proposed shortcut is described ahead: using a steering vector as a deliberate adversarial probe, on the logic that a behavior easily triggered by a small internal nudge might also surface in deployment, while one that resists even that kind of nudge is better evidence of a model's safety.
The same vector, aimed the other way
Every other application in the paper uses a CAA steering vector to suppress or encourage a behavior in the direction alignment wants: less sycophancy, more refusal. The proposed red-teaming application, sketched in the paper's future-work section, runs the identical machinery toward the opposite end - using CAA "as an adversarial intervention to trigger unwanted behaviors in models more efficiently" than prompting alone (contrastive-activation-addition, §"9.1 Suggested future work", p. 9), building on the authors' own prior work applying activation engineering to red-teaming (Panickssery, 2023a). Nothing about the method changes: it is still a mean-difference steering vector added to the residual stream at a chosen layer and multiplier. Only the sign of the intent does - instead of steering a chat model away from an unsafe behavior, the vector deliberately steers it toward one.
Skipping the search for a prompt
Jailbreaks are the paper's evidence that this search is worth automating: users routinely find inputs that defeat a model's safety training, but incidentally, through ordinary use, not through any systematic process. Red teaming elsewhere in this corpus means searching for adversarial prompts directly, whether by crowdworkers or, in Constitutional AI's mention of automated red-teaming, by another language model generating candidate prompts (constitutional-ai, §"A Harmless but Non-Evasive (Still Helpful) Assistant", p. 4). Both still search the space of natural-language strings; an LM-generated attack still has to compose a prompt and hope it works. This proposal removes the search altogether: instead of writing or generating a string that might trigger a behavior, it perturbs the model's internal activations directly with a vector and asks whether the behavior comes out anyway.
What triggering, and failing to trigger, is meant to show
The application is framed as diagnostic rather than harmful in itself. If a behavior can be easily triggered through a small internal perturbation, the paper argues, that is evidence it may also be reachable in deployment through some route a real adversary might find; conversely, an inability to elicit it this way is offered as a stronger safety guarantee than failing to find it by sampling ordinary outputs, since perturbing activations searches part of a model's behavior space that prompting alone cannot easily reach (contrastive-activation-addition, §"9.1 Suggested future work", p. 9). This is presented only as suggested future work, not a result the paper demonstrates - the strength of that guarantee is asserted here, not empirically shown.