Quantitative vs. Qualitative Evaluation Methodology — prefigures → Instruction following (as a fine-tuning objective)

For tasks with no reward function at all, the 2017 paper's only recourse is to 'start from a goal expressed in natural language, ask a human to evaluate the agent's behavior based on how well it fulfills that goal, and then present videos of agents attempting to fulfill that goal' (deep-rl-human-prefs, §"2.1 Setting and Goal", p. 4). That is already the shape of instruction following as InstructGPT later defines and trains it: a natural-language statement of intent, with no formal specification of correctness, evaluated by whether a human judges the output to have satisfied it (instructgpt, §"1 Introduction", p. 1). The domains could hardly be more different — a video of a simulated robot against a block of generated text — but the underlying wager is identical in both papers: that human judgment of goal-fulfillment can stand in for a reward function or loss nobody knows how to write down for the task in question. What changed by 2022 is that this 'qualitative' evaluation stopped being a fallback used only when nothing else was available and became the primary training signal for an entire class of systems, rather than a one-off way of grading backflips and traffic-following.