Elicitation as audit: steering for red-teaming
Knowing that a model was trained to refuse something is not the same as knowing it cannot be made to do it anyway, and the usual way of finding out, people hunting for a prompt that breaks the model, is slow and unsystematic. This theme now covers two different ways of turning that same worry into an audit, both routed through a model's internals rather than through its inputs.
One paper's route is behavioral: deliberately steer the model's activations toward an unwanted behavior and see how easily it moves, on the reasoning that a behavior a small internal push can trigger is plausibly one some prompt could trigger too. A second, newer paper proposes a structural route instead, enumerating and inspecting a model's internal features directly, on the premise that a complete catalogue of what a model represents could itself rule out dangerous behavior without ever having to elicit it.
Validating that finetuning and RLHF have made a model robustly safe is difficult, because unwanted behaviors that survive training can still be hard to find. The theme now holds two distinct routes to that audit. CAA's route is behavioral elicitation: jailbreaks, the adversarial or unusual inputs users find by manual search to make a model output content its safety training was meant to prevent, motivate red teaming, the broader practice -- elsewhere in the corpus a crowdworker-driven activity -- of probing a model for unwanted behavior. CAA proposes turning steering itself into that probe: deliberately push a model's internal activations toward an unwanted behavior to see how easily it can be triggered, on the premise that a behavior easily elicited by a small internal perturbation is plausibly also reachable by some input in deployment, so failure to elicit it becomes a stronger guarantee than failure to find it by sampling outputs. Cunningham et al.'s (2023) sparse-autoencoder paper proposes a second, structural route: enumerative safety, the aspiration that a complete, human-understandable list of a model's features could itself guarantee it will not perform dangerous behaviors such as deception -- audit not by eliciting behavior but by enumerating and inspecting the parts that could produce it.