Two or more remedies for the identical problem, sorted by where they intervene
A single vulnerability sometimes gets two or three answers instead of one, each pitched at a different layer or timescale rather than competing to be correct. Reward tampering, for instance, gets solved once by looking ahead and once by freezing the reward outright.
Adversarial defenses working at different layers and exploration algorithms blind to danger for different reasons follow next, before the same forking shows up at the scale of whole research programs: distant supervision making each check cheaper, hierarchical RL making checks rarer instead.
A recurring shape across two different sections of Concrete Problems: the paper offers not one answer to a vulnerability or an oversight gap but a fork, with each branch distinguished by layer, timescale, or lineage rather than by which one is correct. Model lookahead and reward pretraining solve the same reward-tampering problem by opposite strategies, foresight versus freezing. Adversarial reward functions operates at a different layer than counterexample resistance despite both borrowing adversarial ML. Epsilon-greedy and R-max are blind to danger for opposite reasons. H-infinity control reaches the same conservative envelope as reachability analysis by different mathematics. For scalable oversight, the same forking recurs at the level of whole research programs: distant supervision (instantiated by DeepDive) and hierarchical reinforcement learning (tracing its lineage to feudal RL) are offered as structurally different routes, one making each check cheaper, the other making checks rarer, and apprenticeship learning borrows IIRL's technique while aiming at a different output entirely.