Multiple-Choice Comparison Format — turns into a preference labeler → Feedback Model

feedback-model is defined in the paper as nothing more exotic than "an independent model, typically a pretrained LM" (constitutional-ai, §"4.1 Method", p. 10) — it has no built-in notion of preference or comparison. multiple-choice-evaluation-format is what manufactures that capability: by presenting the conversation, a sampled constitutional principle, and two labeled candidate responses as an (A)/(B) question and reading off the model’s answer log-probabilities, it repurposes ordinary next-token prediction into a scalar harmlessness judgment. This is the specific engineering move that lets a plain, off-the-shelf language model stand in for a crowdworker: without this templating, there would be no way to extract a comparable preference signal from a model that was never trained to produce one.