Adversarial blinding — borrows from a different adversarial ml lineage than → Counterexample resistance

explored within the theme Borrowing adversarial ML to defend against reward hacking

Both proposals are labeled "adversarial" and sit next to each other in the paper's list of remedies, which invites conflating them, but they borrow from two different corners of adversarial machine learning. Counterexample resistance descends from the adversarial-examples literature: it defends a model's output against worst-case perturbations of its input, using adversarial training or weight uncertainty (concrete-problems, §"Counterexample Resistance:", p. 10). Adversarial blinding descends from a different lineage, domain-adversarial and invariance training, where an adversarial objective is used not to robustify a prediction but to strip a model of mutual information with some variable altogether, described in the paper as making it impossible for an agent to understand, or even statistically correlate with, part of its environment, such as how its own reward is generated (concrete-problems, §"Adversarial Blinding:", p. 10). One hardens a model against being fooled; the other prevents the model from perceiving the thing it might exploit in the first place. The paper's own gloss on blinding, "cross-validation for agents," underlines that it is doing something structurally unlike counterexample resistance.