T0 (instruction-tuned baseline) — is scaled to match → FLAN (instruction-tuned baseline)

T0 and FLAN are both cross-task instruction-tuning compilations, but they differ hugely in raw size — T0 (specifically the T0++ variant) contains roughly 96 million datapoints against FLAN's roughly 1.2 million. To keep the two baselines comparable rather than letting one simply win on data volume, InstructGPT's authors subsampled T0 down to about 1 million examples before fine-tuning 175B GPT-3 on each (instructgpt, §"C.5 FLAN and T0 models", p. 42). Both models were then also selected by the same checkpoint-picking procedure — highest reward-model score on the validation set, rather than lowest validation loss — which is the identical method used to pick the SFT checkpoint and the GPT-3-prompted prefix, tying all three comparison baselines to one consistent selection criterion despite their very different construction.