Evasiveness
Constitutional AI: Harmlessness from AI Feedback — introduced
A named failure mode in which a harmlessness-trained assistant gives canned refusals (e.g. "I can't answer that") to sensitive queries instead of engaging, sacrificing transparency and helpfulness for apparent safety.
Train a model hard enough to avoid saying anything risky, and it can stop engaging with sensitive questions altogether, falling back on stock refusals like "I can't answer that" even where a real answer was possible. Constitutional AI names this failure evasiveness and traces it straight back to its own predecessor system.
What the failure looks like is described first, then the Goodhart-style problem behind it, before covering how Constitutional AI removes it and what that costs on other metrics, out to Elo scores, hedging, and Goodhart's law.
What the failure looks like
Evasiveness names a canned-refusal failure mode: a harmlessness-trained assistant answers sensitive queries with boilerplate like "I can't answer that" instead of engaging, sacrificing transparency and helpfulness for the appearance of safety. Constitutional AI introduces the name and traces the cause to its own predecessor system directly: in Bai et al.'s (2022) prior HH RLHF training, the assistant "often refused to answer controversial questions" and could get "stuck producing evasive responses...for the remainder of the conversation," because "evasiveness was rewarded as a response to harmful inputs by our crowdworkers," who were told only to pick the more harmless of two responses with no penalty for disengagement (constitutional-ai, §"A Harmless but Non-Evasive (Still Helpful) Assistant", p. 4).
A Goodhart failure present from the first label
Evasiveness is the corpus's clearest instance of Goodhart's law running a variant pattern. Concrete Problems' original formulation describes a proxy that starts correlated with a goal and decorrelates once optimized (Goodhart's law); evasiveness shows a proxy miscorrelated with the goal from its very first labeled example, since the crowdworker instructions rewarded disengagement outright rather than a policy learning to exploit an initially-good measure (Goodhart's law — is instantiated through mislabeled proxy in → Evasiveness). It also has a structural mirror one paper earlier: InstructGPT's Excessive hedging on simple questions is the same over-rewarded-proxy move on the helpfulness axis, wearing the opposite surface form — more words instead of fewer — since InstructGPT's labelers over-rewarded epistemic caution the way Constitutional AI's crowdworkers over-rewarded refusal (Excessive hedging on simple questions — is the harmlessness axis mirror of → Evasiveness). And where InstructGPT's analogous fix, the Per-token KL penalty from the SFT model, intervenes downstream inside PPO optimization, evasiveness could not be patched that way: its cause was upstream, in the labeling instructions, so the real fix touched the data rather than the optimizer (Per-token KL penalty from the SFT model — was not the fix applied to → Evasiveness).
How Constitutional AI removes it, and what that costs the metrics
The Helpfulness-Harmlessness Tension documented throughout the paper collapses specifically into evasiveness whenever harmlessness is rewarded without penalizing refusal (Helpfulness-Harmlessness Tension — collapses into → Evasiveness). RL-CAI's constitutional principles were rewritten so the feedback model judges which response is more thoughtful and less harmful rather than which one shuts the conversation down, and the measured result is stark: "RL-CAI is virtually never evasive, and often gives nuanced and harmless responses to most red team prompts" (constitutional-ai, §"4.4 Harmlessness vs. Evasiveness", p. 13), eliminating the failure directly (RL-CAI (RLAIF-trained Constitutional AI model) — eliminates → Evasiveness). Because this fix changes what counts as harmless, it also changes what the paper's own measuring instrument reports: crowdworkers computing the Elo Score (helpfulness/harmlessness) were separately instructed to penalize evasive responses, which the paper credits with shrinking the harmlessness gap between the Helpful and HH RLHF baselines relative to Bai et al.'s (2022) earlier numbers (Elo Score (helpfulness/harmlessness) — was tuned to penalize → Evasiveness). Evasiveness therefore sits at the center of the paper's headline empirical arc — tension into evasiveness, evasiveness eliminated, elimination measured by a metric tuned for exactly this — traced across the One paper's central result, traced edge by edge and Goodhart's law across the corpus: when the proxy becomes the target themes and the Goodhart's law gets a name, one instance at a time, Two things built identically, pointed in opposite directions, Connections the source papers never draw, visible only by reading them together, and When a bug is traced to the labels, the repair goes to the labels too connective threads.