Frame problem
Concrete Problems in AI Safety — inherited
A classical AI/knowledge-representation problem (McCarthy & Hayes 1969) concerning the difficulty of efficiently specifying which facts remain unaffected by an action.
How do you write down everything an action does not change? Posed by McCarthy and Hayes in 1969, the frame problem is a classical puzzle of knowledge representation, decades older than machine learning safety.
What an action leaves untouched is the original puzzle; the same gap resurfacing, unresolved, inside modern reward design is the return trip. Negative side effects, the concept whose baseline-distance approach it quietly complicates, is where the thread leads next.
Specifying what an action leaves unchanged
The frame problem, posed by McCarthy and Hayes in 1969, is a foundational difficulty in classical knowledge representation: given a formal description of an action's effects, how does a reasoning system efficiently represent everything the action does not change? In situation calculus, one might describe the effect of an action $a$ on a fluent $f$ by stating when $\mathrm{Holds}(f, \mathrm{do}(a,s))$ becomes true or false, but without additional machinery, nothing in that description tells the system that every other fluent unaffected by $a$ still holds in the successor situation — a system must either write explicit "frame axioms" for every fluent-action pair that does not interact (an enumeration that grows unmanageably with the size of the world model) or find some more compact, general solution. This is a decades-old, unresolved difficulty in AI, predating this paper by nearly fifty years, and Concrete Problems cites it only in a footnote (concrete-problems, §"3 Avoiding Negative Side Effects", p. 4).
Resurfacing, unaddressed, in reward design
The edge Frame problem — supplies the classical diagnosis for → Negative side effects (avoiding) draws the connection explicitly: "Intuitively, this seems related to the frame problem, an obstacle in efficient specification for knowledge representation" (concrete-problems, §"3 Avoiding Negative Side Effects", p. 4). Negative side effects (avoiding) restates the frame problem as an engineering difficulty for reward design rather than logical representation — a designer cannot feasibly enumerate every "vase" an agent must not knock over, because doing so requires specifying everything the task's objective should leave untouched, exactly the kind of exhaustive enumeration the frame problem showed is intractable to write down directly. Concrete Problems does not claim to solve the frame problem; it treats the footnote as a hedge, an acknowledgment that a specification difficulty first identified in symbolic AI planning resurfaces, essentially unaddressed, in modern reinforcement learning objective design. The concept sits in the Penalizing side effects by distance from a baseline theme and the A decades-old, informal idea resurfaces as an unsolved precursor connective theme, and belongs to the Bounding what an agent can change or explore, not just what it's told to want supertheme as the classical-AI ancestor of the paper's baseline-distance strategy for side effects, embodied downstream in the Impact regularizer (defined).