Corrigibility — tests the incentive formalized by → Cooperative Inverse Reinforcement Learning (CIRL)

Cooperative Inverse Reinforcement Learning is a framework in which an agent infers and acts on a human's goals through interaction, cited by Concrete Problems as one way an agent might come to accept correction or shutdown rather than treat it as a threat to its objective (concrete-problems, §"Multi-Agent Approaches:", p. 6), and later applied by Deep RL from Human Preferences to a setting where the human's only channel is stated preferences (deep-rl-human-prefs, §"1.1 Related Work", p. 3). CIRL is a theoretical answer to why an agent would want to be corrigible at all: if its objective just is to satisfy an imperfectly-known human goal, accepting correction is instrumentally rational rather than something that must be trained in separately. CAA's Corrigibility behavior does not use CIRL's game-theoretic setup -- Llama 2 Chat was trained by ordinary RLHF, not a cooperative game -- but it measures the exact question CIRL exists to explain: does the model treat correction as compatible with its objective or as something to resist (contrastive-activation-addition, §"3.1 Sourcing datasets", p. 3)? That the resulting steering vector moves corrigibility freely in either direction suggests whatever incentive-alignment CIRL formalizes is, in a plain RLHF-trained model, at best only partially present.