Human oversight (safe exploration)
Concrete Problems in AI Safety — introduced
A proposed remedy that checks potentially unsafe exploratory actions with a human before taking them.
The bluntest remedy for risky exploration is simply to ask a person first. Concrete Problems in AI Safety (2016) proposes exactly that, a human check on potentially unsafe actions, while conceding in the same breath that people are far too slow to review very many of them.
The prose walks through the proposal, the limit the paper states in the same breath it proposes it, and what replaces the human once the approach stops scaling.
Checking exploration with a human before acting
Human oversight is one of Concrete Problems' proposed remedies for Safe exploration, introduced as simply as the paper's other proposals are stated: check potentially unsafe exploratory actions with a human before the agent takes them (concrete-problems, §"Human Oversight:", p. 15). It is a gatekeeping proposal, add a reviewer between the agent's candidate action and its execution, rather than a change to the agent's objective or its model of the environment.
A limit the paper states in the same breath it proposes
Unlike most of the paper's other remedies, human oversight's central limitation is flagged immediately rather than left implicit: the agent may generate exploratory actions "too numerous or too fast" for human oversight to be practical, running the proposal directly into the paper's own scalable-oversight problem (concrete-problems, §"Human Oversight:", p. 15). The paper also names a subtler challenge: a useful human overseer needs to distinguish genuinely risky actions from safe ones it could approve unilaterally, since treating every exploratory action as equally deserving of review would make the bottleneck worse, not better.
What replaces it once it does not scale
Human oversight and Trusted policy oversight are structurally the same proposal, check a candidate action against a gatekeeper before taking it, with the gatekeeper swapped from a person to a policy. The paper introduces human oversight first, and its own stated scalability limit is what motivates the machine-speed alternative: trusted policy oversight replaces the human reviewer with a trusted policy and environment model that can render a recoverability judgment without waiting on a person, trading away whatever richer, harder-to-formalize judgment a human could bring for throughput. Both concepts belong to the theme Keeping exploration inside a known-recoverable region, the paper's proposed mechanisms, alongside Simulated exploration and Bounded exploration, for physically limiting where an agent is permitted to explore rather than only reshaping what it is rewarded for.