Open-ended generation evaluation with GPT-4 rating — shares the ai judge move with → RL from AI Feedback (RLAIF)

Both are answers to the same bottleneck: once a paper needs judgments at a scale humans cannot cheaply supply, it substitutes a capable model for the human judge. Constitutional AI's RLAIF generates the harmlessness preference labels that train its preference model by showing a feedback model a prompt and a pair of responses and taking its multiple-choice judgment, replacing the human crowdworkers used for the same paper's helpfulness labels (constitutional-ai, §"4 Constitutional AI: Reinforcement Learning from AI Feedback", p. 10). CAA's open-ended evaluation makes the same substitution one step later in the pipeline: instead of training a policy on AI-generated labels, it uses GPT-4 directly as the measurement instrument, scoring how strongly a generation displays a targeted behavior against natural-language rubrics (contrastive-activation-addition, §"4.2 Open-ended generation", p. 4). Neither paper treats this as costless -- RLAIF still sources helpfulness labels from humans, and CAA still manually spot-checks its GPT-4 scores -- but both accept an AI judge as the only way to close the loop between a generation and a scalar score of how well it matches an intended property, once the needed labels are simply too many for a person to produce.