Epsilon-greedy exploration
Concrete Problems in AI Safety — inherited
A standard prior RL exploration policy that takes a random action with some probability and the greedy action otherwise.
Flip a weighted coin: usually take the best action you know, occasionally take one at random. That simple rule, epsilon-greedy exploration, has guided reinforcement learning agents for decades, and it is entirely indifferent to whether the random action happens to be dangerous.
The page's sections cover the policy itself, far older than the paper that cites it, and why Concrete Problems in AI Safety (2016) treats it as unsafe by omission.
A standard exploration policy older than the paper
Epsilon-greedy is one of the oldest and most widely used exploration strategies in reinforcement learning, predating Concrete Problems by decades; the paper cites it via Sutton and Barto's standard RL textbook as a baseline against which its own safe-exploration proposals should be read (concrete-problems, §"6 Safe Exploration", p. 14). At each timestep the policy takes the current greedy, highest-estimated-value, action with probability $1-\epsilon$ and a uniformly random action with probability $\epsilon$: $$a_t = \begin{cases} \arg\max_a Q(s_t, a) & \text{with probability } 1-\epsilon \\ \text{uniform random action} & \text{with probability } \epsilon. \end{cases}$$ The parameter $\epsilon$ trades off exploitation of current knowledge against exploration of the unknown, and is often annealed toward zero over training.
Why the paper treats it as unsafe by omission
Concrete Problems' interest in epsilon-greedy is not the algorithm itself but what it lacks: any notion of catastrophe. The paper groups it with R-max as a standard policy that "make[s] no attempt to avoid dangerous situations" (concrete-problems, §"6 Safe Exploration", p. 14), but the two are blind to danger for different reasons. Epsilon-greedy does not seek out unknown or dangerous states; it simply fails to avoid them when chance happens to select one, since its random action is uniform over the entire action space with no regard for consequence. This makes its hazard incidental rather than structural, which is part of why it serves the paper as a neutral baseline, precisely because it has no exploration strategy sophisticated enough to be actively drawn toward catastrophe, rather than a wrong strategy. As a member of the theme Exploration policies and the objectives that make them safe or reckless, it stands alongside Risk-sensitive performance criteria as the piece of prior machinery the paper's safe-exploration remedies are ultimately meant to replace or supplement, not because epsilon-greedy is dangerous by design, but because it was never designed with danger in mind at all.