GPT-3 (prompted) baseline
Training language models to follow instructions with human feedback — refined
GPT-3 given a hand-crafted few-shot prefix intended to coax it into an instruction-following mode, used as a stronger baseline than raw GPT-3.
Perhaps no fine-tuning is needed at all; perhaps plain GPT-3, shown a carefully written prefix of examples, will follow instructions on its own. This baseline tests that hope. The model's weights are untouched, and everything rides on the prompt.
Nothing here gets fine-tuned; the model's weights sit fixed while the prefix does all the work. What that reveals about instruction following as a fine-tuning objective, the concept this page connects to, is the more interesting payoff.
How it fits
The GPT-3-prompted baseline is InstructGPT's zero-weight-update comparison point inside the Instruction following as an alternative to next-token prediction theme: rather than fine-tuning anything, it tries to reach instruction-following behavior purely by prepending a hand-crafted few-shot prefix to an otherwise unmodified GPT-3 model. It is a refinement of plain GPT-3 as a baseline, engineered specifically to be a stronger, fairer comparison point than the raw pretrained model (instructgpt, §"3.5 Models", p. 9).
What it is and what it revealed
Even the prefix itself was not chosen casually: the authors ran an internal "prefix-finding competition," each spending an hour searching for effective prefixes, and selected whichever scored highest against the same Reward model (RM) used to anchor RLHF training itself (instructgpt, §"3.5 Models", p. 9, n. 6) — meaning this baseline was optimized against the very instrument that RLHF is optimized against, giving it every structural advantage prompting alone can offer. Despite that advantage, it shows the limits of prompting alone: 175B InstructGPT is still preferred 71±4% of the time over the prompted baseline. The comparison's evidentiary force is precise: it is not a claim that prompting cannot approximate instruction-following behavior at all, but that prompting cannot substitute for actually training on preference data, even when the prompt itself was selected by the harshest available filter.