HellaSwag
Training language models to follow instructions with human feedback — inherited
A commonsense-reasoning benchmark (Zellers et al., 2019) requiring selection of the most plausible sentence continuation, used as one of the public NLP evaluation datasets.
Pick the most plausible ending to a short everyday scene: easy for people, and made deliberately hard for machines. HellaSwag's wrong answers are generated and filtered specifically to fool models, which is what keeps this commonsense benchmark discriminating.
Its particular role here is the recoverable half of the alignment-tax story, a capability slip that, unlike some others, bounces back. That distinction is worth more attention than the benchmark's mechanics.
The benchmark
HellaSwag (Zellers et al., 2019) is a commonsense-reasoning benchmark requiring a model to select the most plausible continuation of a short scenario from four candidate endings. Its distinguishing feature is Adversarial Filtering: the wrong-answer distractors are generated by a language model and then filtered to be maximally difficult for machine classifiers while remaining easy for humans, producing endings that are grammatically fluent but subtly implausible. HellaSwag predates InstructGPT by three years and is inherited here as one of the public benchmarks used to measure whether RLHF fine-tuning degrades general commonsense-reasoning capability (instructgpt, §"1 Introduction", p. 3; §"Hellaswag", p. 48), scored by ranking the given candidates by average per-token log probability rather than by generating free text (instructgpt, §"D Automatic evaluation details", p. 44).
Its role: the recoverable half of the alignment tax
The HellaSwag — escapes the regression that persists in → DROP (Discrete Reasoning Over the content of Paragraphs) edge shows HellaSwag on the favorable side of a split the paper draws among its own regressions: PPO-ptx "even surpasses GPT-3 on HellaSwag" while it "still lags behind GPT-3 on DROP, SQuADv2, and translation" (instructgpt, §"4.2 Results on public NLP datasets", p. 15), with the 175B PPO-ptx model reaching 0.807 zero-shot accuracy against GPT-3's 0.781 (instructgpt, Table 14, p. 56). A plausible mechanical reason tracks the split across the whole The public NLP benchmark suite used to catch regressions theme: HellaSwag is scored by likelihood ranking over fixed candidates, never requiring the model to generate text, whereas the tasks that stay regressed, DROP, SQuAD v2, and translation, require sampling free-form output and scoring it against a target by overlap. The recoverable part of the Alignment tax, on this evidence, sat in the model's relative rankings of given options; the persistent part lives in what the model actually chooses to write.