Constant Rater-Error Noise Model — assumes a constant later measured by → Inter-labeler agreement rate

The 2017 paper handles the fact that human raters make mistakes by assumption rather than by measurement: it builds a constant 10% chance of a uniformly random response directly into the preference-prediction softmax, because 'human raters have a constant probability of making an error, which doesn't decay to 0' as the reward difference grows (deep-rl-human-prefs, §"2.2.3 Fitting the Reward Function", p. 5). That figure is not derived from any data the 2017 paper collected about its own contractors; it is a modeling convenience chosen to keep the loss well-behaved. InstructGPT instead measures rater reliability directly, computing an inter-labeler agreement rate from labelers actually judging the same outputs, turning 'how often do humans agree' from an assumed constant into a reported statistic that can vary by task and be used to validate the labeling pipeline itself (instructgpt, §"3.4 Human data collection", p. 8). The two papers answer the same underlying worry — that comparisons are noisy — with opposite epistemic postures: one builds in a fixed error rate so the math stays tractable, the other goes and finds out what the real rate is.