Testing generative capability: translation and summarization benchmarks
Choosing among options is one kind of test; producing fluent, on-topic text from scratch is a stricter one. This theme covers the generation benchmarks where InstructGPT (2022) actually has to produce text rather than select it, French-to-English translation and two summarization tasks, making them some of the clearest evidence in the paper for a real alignment tax.
CNN/Daily Mail and Reddit TL;DR share a scoring method, both judged with ROUGE-L against reference summaries, while the translation task relies on BLEU instead.
Where the language-understanding benchmarks test comprehension, this smaller group tests generation directly, and each carries its own scoring convention. WMT 2015 French-to-English translation evaluates output with BLEU. CNN/Daily Mail summarization and Reddit TL;DR summarization, prompted with a TL;DR: format, are both scored with ROUGE-L against reference summaries. All three are included in InstructGPT's automatic evaluation suite for the same reason as the understanding benchmarks: to check whether RLHF fine-tuning, optimized entirely around human preference on API prompts, degrades performance on tasks with an established public track record and an existing metric. That these tasks require actually producing fluent, on-topic text, not merely selecting among options, makes them a stricter test of whether instruction-tuned capability transfers, and the paper reports measurable regressions on exactly this group as some of its clearest evidence for an alignment tax.