Reddit TL;DR summarization
Training language models to follow instructions with human feedback — inherited
A summarization benchmark (Volske et al., 2017) of Reddit posts with author-written TL;DR summaries, included in the automatic evaluation suite.
Reddit users often end long posts with their own TL;DR, and a 2017 dataset turned that habit into a benchmark: posts paired with author-written summaries, which InstructGPT uses to check that fine-tuning has not dulled its summarization ability.
What it revealed, including a reporting problem shared with its neighbors, matters more than the setup. That shared problem is the thread connecting it to both CNN/Daily Mail summarization and the WMT translation task.
How it fits
Reddit TL;DR summarization (Volske et al., 2017) is a summarization benchmark of Reddit posts paired with author-written TL;DR summaries, inherited into InstructGPT's automatic evaluation suite within Testing generative capability: translation and summarization benchmarks, scored via ROUGE-L against the reference summaries (instructgpt, §"D Automatic evaluation details", p. 44; §"TLDR Summarization", p. 50). It predates InstructGPT and, notably, is the exact domain in which Stiennon et al. (2020) originally developed the RLHF summarization method InstructGPT builds on.
What it revealed, and a reporting problem it shares
Because the API prompt distribution driving RLHF is over 96% English and summarization is a named API use case whose TL;DR-style prompt format mirrors real user traffic — listed at 4.2% of prompts in Table 1 — Reddit TL;DR sits close to the RLHF fine-tuning distribution in a way few other benchmarks do. This proximity produces a striking asymmetry: WMT15 fr-en bears the alignment tax that spares this benchmark. TL;DR ROUGE-L actually rises above the GPT-3 baseline under PPO training, from 0.196 to 0.227, even as French-to-English translation collapses under the same training regime (instructgpt, §"4.2 Results on public NLP datasets", p. 15; Table 14, p. 56). The lesson is that generative capability transferred where the benchmark resembled the RLHF tuning data and was spent where it did not — task difficulty alone does not predict which benchmarks the Alignment tax hits. Separately, this benchmark is the target of a data-integrity problem rather than a measurement one: CNN/DM summarization duplicates its reported results, with the two benchmarks' rows in Table 14 numerically identical across all twelve models to three decimal places, almost certainly a copy-paste error rather than two genuinely independent measurements (instructgpt, §"E.1 Performance on public NLP datasets", p. 56).