Use demonstrations (safe exploration) — operationalized as → Supervised fine-tuning (SFT)

Concrete Problems proposes "use demonstrations" as one of several routes to avoid unsafe exploration: learn a baseline policy from expert trajectories via inverse RL or apprenticeship learning, then explore only around that baseline (concrete-problems, §"Use Demonstrations:", p. 15). InstructGPT's supervised fine-tuning step realizes this proposal almost exactly, but by the simplest possible mechanism the 2016 paper considered: instead of first recovering a reward function or matching feature expectations, as IRL and apprenticeship learning do, SFT fine-tunes GPT-3 directly on demonstrations with ordinary supervised learning, no reward inference or policy-matching machinery involved (instructgpt, §"3.1 High-level methodology", p. 6). The exploration-safety motivation also drops out: SFT is not framed as bounding a later RL exploration phase but as a free-standing baseline and initializer for the stages that follow. What survives is the core wager — that a policy learned from demonstrations can substitute for a hand-specified objective.