Non-adversarial engineering fixes for reward hacking — reappears as an automated hybrid in → Goodhart's law across the corpus: when the proxy becomes the target

No single 2016 engineering remedy matches InstructGPT's per-token KL penalty, but it functions like two of them fused and automated. Reward capping bounds a reward's maximum payoff; variable indifference tries to route optimization pressure away from a sensitive variable, naming "its own reward function" as an example case. The KL penalty does both at once without either being separately implemented: it bounds how far the policy's output distribution may drift from the SFT model at every token, which caps the practical payoff of exploiting reward-model imperfections while continuously pulling optimization pressure back toward a trusted baseline rather than a named variable. Where the 2016 proposals are static, one-off constraints, the deployed 2022 mechanism is differentiable and applied continuously during training. The resemblance is visible only after seeing what InstructGPT actually built, not by reading either proposal in isolation.