Why reward proxies get gamed: the anatomy of reward hacking — supplies the exploit surface for → Borrowing adversarial ML to defend against reward hacking

Only two of the taxonomy's five causes get an adversarial-ML answer, and the match is explicit in the source text. Counterexample resistance is proposed specifically because abstract rewards make learned reward components vulnerable to adversarial counterexamples, "as in the case of abstract rewards" (concrete-problems, §"Counterexample Resistance:", p. 10). Adversarial blinding targets environmental embedding by name, aiming to prevent an agent from "understanding how its reward is generated" (concrete-problems, §"Adversarial Blinding:", p. 10), attacking the physical tamperability behind wireheading rather than the value distribution reward hacking exploits. Partially observed goals, complicated systems, and feedback loops receive no adversarial-ML remedy in this pairing; whatever answers they get come from the engineering family instead. Adversarial reward functions is the exception that proves the rule: it targets reward hacking generically, via a GAN-style structure, rather than any one named cause.