Elo Score (helpfulness/harmlessness) — supplies the explanatory analogy for → Bradley-Terry Model

The paper never computes anything like an Elo rating; it invokes Elo purely as an analogy to explain what the Bradley-Terry softmax in Equation 1 is doing (deep-rl-human-prefs, §"2.2.3 Fitting the Reward Function", p. 5). Just as a difference in Elo rating between two chess players is used to estimate the probability that one player beats the other, the paper treats a difference in predicted reward between two trajectory segments as estimating the probability that a human prefers one segment over the other, with the reward model's output playing the same explanatory role as a chess rating. The analogy is doing real interpretive work: it is what licenses treating a network trained purely on binary comparisons as if it had learned a meaningful interval reward scale, when the training signal only ever committed the network to getting relative comparisons right. That interpretive move, treating comparison outcomes as evidence for an underlying rating-like scale, is exactly what lets a single scalar reward per segment, rather than a full ranking over many segments, be enough information to drive policy optimization.