Limiting potential influence instead of measuring realized change
An agent does not have to change anything to become dangerous; quietly acquiring the power to change everything can be enough. This theme covers a second approach to side effects in Concrete Problems (2016), one that measures potential influence rather than realized harm, borrowing empowerment from information theory to formalize what that potential even means.
The page separates this from baseline-distance penalties before turning to reward uncertainty, a companion idea that manufactures caution not through an explicit penalty but by simply refusing to let the agent be sure what it is being rewarded for.
A second, distinct strategy for avoiding negative side effects does not measure distance from a baseline state at all, and this theme covers it. Penalize influence discourages an agent from putting itself in a position of high potential power over its environment, not merely from acting on that power once acquired, a distinction that matters because an agent could accumulate influence and simply hold it in reserve. The measure behind this proposal is empowerment, an information-theoretic quantity defined as the maximum mutual information between an agent's future actions and future states, borrowed to formalize what potential influence means. Reward uncertainty takes a different tack again, replacing a single fixed reward function with a probability distribution over rewards biased toward assuming unplanned environmental change is bad, so caution emerges from uncertainty rather than an explicit penalty term. Low-impact agents, an informally discussed prior concept, names the general target all of these proposals aim at. This theme's claim is that influence and uncertainty offer an alternative vocabulary to baseline-distance for the same underlying problem.