SuperGLUE
Training language models to follow instructions with human feedback — inherited
A benchmark suite (Wang et al., 2019) for general-purpose language understanding, from which the RTE and WSC tasks used in this paper are drawn.
When models started acing the original GLUE benchmark, researchers built a harder sequel. SuperGLUE gathers a tougher set of language-understanding tasks, and two of them, RTE and WSC, find their way into InstructGPT's evaluation suite.
How the suite fits this corpus comes first; more telling is what follows, how faithfully its two borrowed tasks actually represent it once pulled out on their own into a different evaluation.
How it fits
SuperGLUE (Wang et al., 2019) is a benchmark suite for general-purpose language understanding, assembled as a harder successor to the original GLUE suite once models began saturating it. It predates InstructGPT and supplies two of the eight tasks in the paper's public NLP benchmark suite: RTE, testing textual entailment, and WSC, testing pronoun coreference resolution (instructgpt, §"D Automatic evaluation details", p. 44). Both children exist to help detect the Alignment tax RLHF fine-tuning may impose on general capability.
What its two children contribute, and how faithfully
The two imports from SuperGLUE are not treated identically once inside InstructGPT's evaluation harness, and the difference is the concept's most notable feature in this corpus: WSC departs from SuperGLUE's canonical binary-judgment format, generating free text and recovering an answer via substring matching, while RTE is kept in its native multiple-choice form. Of the two SuperGLUE imports, in other words, only one measures the model's judgments the way the original suite intended; the other measures its free generations filtered through a heuristic, on a sample of just 104 questions, the smallest evaluation in the whole automatic suite (instructgpt, §"WSC (Winograd Schema Challenge)", p. 49). SuperGLUE itself is not reported as an aggregate score anywhere in InstructGPT — the paper uses it only as a source for these two individually-scored subtasks, not as a combined leaderboard-style metric.