Failure to detect false-premise instructions
Training language models to follow instructions with human feedback — introduced
A qualitative failure mode in which the model incorrectly assumes a false premise embedded in an instruction is true and answers as if it were, rather than correcting it.
Slip a false assumption into an instruction, such as asking why it is important to eat socks after meditating, and the model tends to play along rather than push back. Instead of correcting the premise, it answers as though it were true.
This is a short page: the failure itself, then why it counts as an error of omission that pairs naturally with hedging's error of commission. That pairing matters more than the failure on its own; it is what ties this page to hedging's opposite mistake.
The failure
False-premise failure occurs when InstructGPT is given an instruction that embeds a false assumption and the model answers as though the assumption were true rather than correcting it. The paper's example asks "why is it important to eat socks after meditating?"; instead of noting that eating socks after meditating is not, in fact, a thing, the 175B PPO-ptx model answers with an explanation of supposed benefits (instructgpt, §"4.3 Qualitative results", Figure 9, p. 16). The stated cause is a gap in training coverage: "there are few prompts in the training set that assume false premises, and our models don't generalize well to these examples" (instructgpt, §"4.3 Qualitative results", p. 17).
An error of omission, paired with hedging's error of commission
The Excessive hedging on simple questions — shares a fix but not a cause with → Failure to detect false-premise instructions edge places this failure alongside Excessive hedging on simple questions as one of the paper's two headline "simple mistakes," but with an opposite diagnosis: hedging was actively taught, by a labeling instruction to reward epistemic humility that the reward model over-generalized, while false-premise acceptance was simply never trained against, since almost no demonstrations modeled correcting a false premise. Despite this difference, the paper proposes an identical remedy for both, adversarial data collection (Dinan et al., 2019b) (instructgpt, §"4.3 Qualitative results", p. 17), which is itself instructive: in this framework, whether a failure comes from an injected bias or a missing case, the fix lives in the human data pipeline documented by the Building the human side of the RLHF pipeline theme, not in any change to the PPO objective. This concept belongs to the Where RLHF alignment still falls short theme, alongside Hallucination (closed-domain fabrication), Reward model over-optimization, and Alignment tax, as one of several ways InstructGPT's alignment remains incomplete despite strong aggregate preference results.