HellaSwag — escapes the regression that persists in → DROP (Discrete Reasoning Over the content of Paragraphs)

Both benchmarks appear on the paper's original casualty list of RLHF regressions (instructgpt, §"1 Introduction", p. 3), but they part ways under the fix: PPO-ptx "even surpasses GPT-3 on HellaSwag" while it "still lags behind GPT-3 on DROP, SQuADv2, and translation" (instructgpt, §"4.2 Results on public NLP datasets", p. 15). The mechanical difference between the two tasks is a plausible reason. HellaSwag is scored by ranking the given candidate endings by average per-token log probability — the model never generates text — whereas DROP requires sampling a free-form answer at temperature zero and scoring its token overlap with the target (instructgpt, §"D Automatic evaluation details", p. 44). Across the suite the split repeats: the multiple-choice accuracy tasks recover to or above GPT-3 under PPO-ptx (HellaSwag 0.807 versus 0.781 zero-shot at 175B), while the tasks requiring sampled text and overlap scoring stay below baseline (Table 14, p. 56). The recoverable part of the alignment tax, in other words, sat in the model's likelihood rankings; the persistent part lives in what the model actually chooses to write.