Likert overall-quality rating
Training language models to follow instructions with human feedback — inherited
A 1-7 scale on which labelers rate the overall quality of a single model output, collected alongside pairwise rankings and metadata.
Alongside every ranking, labelers also scored outputs on their own merits: a plain one-to-seven quality scale, judged in isolation rather than by comparison. Likert scales are ordinary survey machinery, borrowed here rather than invented.
An absolute score reveals things a head-to-head comparison cannot, which is the more interesting half of this page. Win rate, its one connection, is worth reading beside it for the contrast.
How it fits
The Likert overall-quality rating is a 1-7 scale on which labelers rate a single model output's quality in isolation, collected through the Web labeling interface alongside pairwise rankings and metadata. It is inherited rather than invented by InstructGPT — Likert-scale rating is a general-purpose survey and evaluation technique that long predates this paper — and sits within Automatic overlap scores versus human and preference judgments as a judgment-based metric alongside Win rate (preference against baseline) (instructgpt, §"3.6 Evaluation", p. 10; §"E.4 Likert scores", p. 53).
What it enables that win rate cannot
The Likert score corroborates win rate, but its value in this corpus is less about confirmation than about covering a structural weakness win rate has by design: every win-rate number is relative to a designated anchor, the 175B SFT baseline, so changing the anchor rescales everything. The Likert scale needs no anchor, since it scores a single output on an absolute range. InstructGPT reports that Likert results largely track the preference results (instructgpt, §"E.4 Likert scores", p. 53), which confirms the headline ordering is not an artifact of the particular baseline chosen. But the Likert score also does independent work win rate structurally cannot: it is the scale used to place the FLAN and T0 baselines against GPT-3 variants (instructgpt, §"4.1 Results on the API distribution", p. 12), it served as the guardrail metric in the pretraining-coefficient ablation, where it proved insensitive to the coefficient's exact value (instructgpt, §"E.6 Fixing regressions on public NLP datasets", p. 53), and the final checkpoints in the learning-rate scans were selected by highest Likert score rather than by win rate (instructgpt, §"E.9 Learning rate optimization for PPO models", p. 58). A metric introduced as secondary corroboration ends up quietly choosing which model checkpoints actually ship.