Bradley-Terry Model — is carried over unchanged into → Reward model pairwise comparison loss

The 2017 paper's loss function is not a bespoke invention; it is explicitly named as 'the Bradley-Terry model...for estimating score functions from pairwise preferences,' with predicted reward summed over a segment and passed through a softmax to yield a preference probability (deep-rl-human-prefs, §"2.2.3 Fitting the Reward Function", p. 5). InstructGPT's reward model loss is the same mathematical object five years later: a cross-entropy loss over pairs of ranked completions in which the reward difference between preferred and dispreferred outputs represents the log odds a human would prefer the former (instructgpt, §"3.5 Models", p. 8). Nothing about the functional form changed between the two papers — no new term was added, and the softmax-over-summed-reward structure is identical; the only difference is what fills the two slots being compared, a robot's trajectory segment in 2017 or a language model's completion in 2022. Neither paper cites the other on this point, and InstructGPT never names Bradley-Terry at all, so the identity of the two loss functions is a fact about the mathematics rather than a claimed lineage — a 1952 statistical model for ranking, adopted for deep RL in 2017, still doing the exact same job unmodified in a 2022 language-model reward model.