Excessive hedging on simple questions — shares a fix but not a cause with → Failure to detect false-premise instructions

The paper presents these two failures side by side as "simple mistakes" (instructgpt, §"4.3 Qualitative results", p. 16), but its diagnosis splits them into opposite categories. Hedging is an error of commission by the training signal: labelers were instructed to reward epistemic humility, the reward model absorbed that preference, and the policy learned to overdo it. False-premise acceptance is an error of omission by the data: few training prompts contained false premises, so the model never learned to push back (instructgpt, §"4.3 Qualitative results", p. 17). One failure was actively taught; the other was simply never trained against. Yet the paper proposes the same remedy for both — adversarial data collection — which is what makes the pairing instructive: whether the human data pipeline injected a bias or left a gap, the correction happens in the same place, the data, not the algorithm. Neither failure prompts any proposed change to the PPO machinery. In this framework, alignment bugs are data bugs, and the fix for a mis-specified preference and a missing preference is identical in kind.