Borrowing adversarial ML to defend against reward hacking — needs a more powerful referee than → Non-adversarial engineering fixes for reward hacking

The two remedy families differ in what they presuppose, not just in technique. Adversarial reward functions requires building a second learned system explicitly more capable than the one it polices: the paper concedes "we would have to ensure that the reward-checking agent is more powerful (in a somewhat subtle sense) than the agent that is trying to achieve rewards" (concrete-problems, §"Adversarial Reward Functions:", p. 9), a dependency the proposal never discharges. Counterexample resistance inherits the same trust problem, since it hardens a learned component rather than removing the need to trust one. The engineering family avoids this: sandboxing, capping, multiple independent implementations, trip wires, and pretraining a reward function before interaction all work without positing any second agent smarter than the first. The adversarial family buys generality at the price of a capability assumption the engineering family never has to make.