WMT 2015 French-to-English translation
Training language models to follow instructions with human feedback — inherited
A machine-translation benchmark (Bojar et al., 2015) used as one of the public NLP datasets showing performance regressions under PPO fine-tuning.
Translation is one of the oldest yardsticks in language processing, and InstructGPT keeps one in its kit: a French-to-English task from the 2015 WMT shared translation campaign, scored against human reference translations.
What the numbers actually showed under fine-tuning matters more than the task's mechanics, and Reddit TL;DR summarization, its connected neighbor, is worth reading alongside it: a very different genre, facing the same test.
How it fits
WMT 2015 French-to-English translation (Bojar et al., 2015) is a machine-translation benchmark, scored with BLEU, inherited into InstructGPT's generation benchmark group to test whether RLHF fine-tuning degrades generative capability with an established public track record (instructgpt, §"1 Introduction", p. 3; §"WMT Fr En 15", p. 49).
What it revealed
This benchmark supplies the corpus's cleanest illustration that the Alignment tax tracks distance from the RLHF fine-tuning distribution rather than task difficulty: it bears the tax that spares Reddit TL;DR summarization, even though both tasks demand fluent, open-ended generation. PPO drops 175B zero-shot BLEU on WMT from 38.92 to 24.16, and even PPO-ptx, the paper's dedicated fix for the alignment tax, recovers only to 34.28 — translation stays on the short list of regressions the paper concedes are not fully fixed, alongside DROP and SQuAD v2 (instructgpt, §"4.2 Results on public NLP datasets", p. 15). The explanation is distributional rather than about task difficulty: the API prompt data driving RLHF is over 96% English (instructgpt, §"3.3 Tasks", p. 7), so French-to-English translation is essentially unrepresented in the fine-tuning distribution, while summarization is both a named API use case and the exact domain in which the underlying RLHF method was originally developed by Stiennon et al. (2020). Generative capability transferred where the evaluation benchmark resembled the training data and was spent where it did not — the clearest single piece of evidence in the corpus that the alignment tax is a distributional artifact, not a uniform tax on generation quality.