Adversarial examples — motivates but only partially covers → Counterexample resistance

Counterexample resistance is a direct transplant: harden a learned reward model against exploitation the same way adversarial training and weight uncertainty harden an image classifier against adversarial examples, inputs with imperceptible perturbations that cause confidently wrong predictions (concrete-problems, §"Counterexample Resistance:", p. 10). The transplant is explicitly partial, and the paper says so. Adversarial examples are a well-defined, narrow vulnerability: a classifier's output changes sharply under a small, targeted input perturbation. Reward hacking is broader than that; a cleaning robot pouring out bleach to game a proxy, or an ad system caught in a feedback loop, involves no perturbed input and no misclassification anywhere. The paper is explicit that "adversarial counterexamples are just one manifestation of reward hacking, so counterexample resistance can only address a subset of these potential problems" (concrete-problems, §"Counterexample Resistance:", p. 10). Counterexample resistance is therefore best read as covering only the abstract-rewards slice of the taxonomy, where a reward function is itself a learned high-dimensional model and can literally have adversarial examples in the classifier sense.