Comparisons vs. Absolute Scores (Target Ablation) — swaps in regression in place of → Bradley-Terry Model

The comparisons-vs-absolute-scores ablation directly targets the Bradley-Terry model's own loss: rather than fitting the reward predictor to cross-entropy over pairwise comparisons, it lets an oracle supply the true total reward over a segment and fits the predictor to that value with mean squared error instead (deep-rl-human-prefs, §"3.3 Ablation Studies", p. 9). The result complicates any simple story that comparisons are strictly better: on the continuous control tasks, predicting comparisons clearly outperformed regressing to a score, apparently because the varying scale of raw rewards makes direct regression harder, a difficulty the comparison loss smooths away since it never needs to match an absolute magnitude (deep-rl-human-prefs, §"3.3 Ablation Studies", p. 10). On Atari, where rewards were clipped so the target regression effectively only had to predict a sign, comparisons and score-regression instead performed differently across games with neither consistently ahead. So the paper's own ablation shows the Bradley-Terry framing is not obviously necessary for a clipped, sign-only reward signal, only for continuous magnitudes.