API prompt distribution — supplies label free scale to → PPO dataset

The three training datasets embody a declining gradient of human effort per prompt: SFT prompts need a full written demonstration, RM prompts need a ranking of up to nine outputs, and PPO prompts need nothing at all. The PPO dataset is the only one drawn purely from the API, with no labeler-written prompts (instructgpt, §"3.2 Dataset", p. 7) — by stage three the bootstrap problem is solved and the raw distribution can be used as-is, because RL rollouts generate their own training signal via the reward model. This is where the economics of RLHF show up concretely: 31k unlabeled prompts support 256k training episodes, roughly eight passes over the prompt set (instructgpt, §"C.4 Details of RLHF training", p. 42), while the expensive human stages stay capped at 13k demonstrations and 33k rankings. The reward model functions as an amplifier that converts a bounded stock of human judgment into unbounded per-rollout feedback — the PPO dataset is deliberately the cheapest input in the pipeline because everything costly has been front-loaded into the model that scores it.