Constant Rater-Error Noise Model — corrects the asymptotics of → Bradley-Terry Model

Applied directly, the Bradley-Terry softmax in Equation 1 predicts that a human's probability of preferring the higher-reward segment approaches 1 as the gap between the two summed rewards grows arbitrarily large. The rater-error noise model rejects that asymptotic behavior: it assumes a constant 10% chance the human answers uniformly at random regardless of how large the reward gap is, capping the model's confidence in any comparison well below certainty (deep-rl-human-prefs, §"2.2.3 Fitting the Reward Function", p. 5). The paper's stated reason is empirical rather than mathematical convenience: human raters have a genuine, non-vanishing error rate that does not shrink to zero on easy comparisons, so a model that lets predicted confidence go to 1 would be systematically overconfident. This turns Equation 1 from a pure softmax into a mixture between that softmax and a uniform distribution, a strictly weaker and better-calibrated preference predictor for the same underlying reward difference.