Model Calibration (well-calibrated multiple-choice probabilities) — is the precondition for → Multiple-Choice Comparison Format

multiple-choice-evaluation-format converts a feedback model’s ordinary next-token log-probabilities on "(A)" and "(B)" into a soft preference label — but that move is only meaningful because of model-calibration: prior work (Kadavath et al., 2022) established that language models’ probabilities on multiple-choice questions are reasonably trustworthy estimates, not arbitrary numbers. The paper leans on this directly: "we make a labeled, preference modeling comparison example with the normalized probabilities as targets... we expect these targets will be fairly well-calibrated... since they are multiple choice responses" (constitutional-ai, §"4.1 Method", p. 10). Without model-calibration holding, there would be no principled reason to prefer these soft probability targets over hard 0/1 labels, and the justification for treating a plain LM’s raw output as usable training signal would collapse.