Model lookahead — solves the same problem by a different route than → Reward pretraining

Both remedies target the same underlying vulnerability, an agent's ability to influence the reward function it is being optimized against, but they resolve it by opposite strategies. Model lookahead keeps the reward function mutable and interactive; it instead scores anticipated future states rather than the present one, so that a plan to overwrite the reward function earns negative reward before the overwrite happens rather than after, when it would be too late to matter (concrete-problems, §"Model Lookahead:", p. 9). Reward pretraining removes mutability altogether: it fixes the reward function ahead of time via supervised learning, divorced from any interaction with the environment the agent will later act in, explicitly offered "as a possible defense against cases where the agent can influence its own reward function (e.g. feedback or environmental embedding)" (concrete-problems, §"Reward Pretraining:", p. 10). Lookahead bets on foresight inside an evolving reward signal; pretraining bets on freezing the signal so there is nothing left to influence, at the cost of being unable to keep improving the reward function after deployment.