Long-term / superintelligence AI-risk framing
Concrete Problems in AI Safety — inherited
The pre-existing public discourse (e.g. Bostrom's Superintelligence, MIRI, FHI) that frames AI risk in terms of extreme, speculative scenarios involving highly advanced or superintelligent agents.
Before Concrete Problems appeared in 2016, most public discussion of AI risk revolved around speculative scenarios about superintelligent machines -- the world of Bostrom's Superintelligence, MIRI, and FHI. This page is about that earlier framing, the backdrop the paper wrote against.
Concrete Problems positions itself against this discourse in a move better read as a change of level than an outright rejection, and the pages ahead explain why. From there the trail runs back to the accidents framing that grew out of the disagreement.
The discourse Concrete Problems positions itself against
Long-term or superintelligence-focused AI-risk framing is the pre-existing public and academic discourse, associated with Nick Bostrom's writing on superintelligence, the Future of Humanity Institute, and the Machine Intelligence Research Institute, that discusses AI risk in terms of extreme, speculative scenarios involving highly advanced or superintelligent future agents. This framing predates Concrete Problems and treats questions like value misspecification and loss of control at large scale as its central concern, typically reasoning about capabilities and motivations well beyond present-day systems. The paper inherits and directly engages this framing rather than introducing it, citing the Future of Humanity Institute and Machine Intelligence Research Institute by name (concrete-problems, §"8 Related Efforts", p. 20).
A deliberate change of level, not a rejection
Accidents in machine learning systems — reframes away from → Long-term / superintelligence AI-risk framing documents how the paper positions itself relative to this discourse: not as a refutation, but as a change of method. Section 1 states that "one need not invoke these extreme scenarios to productively discuss accidents, and in fact doing so can lead to unnecessarily speculative discussions that lack precision" (concrete-problems, §"1 Introduction", p. 2), and Section 8 credits FHI and MIRI with studying advanced-AI safety "at a more philosophical level" while positioning its own contribution as "the empirical study of practical safety problems in modern machine learning systems" (concrete-problems, §"8 Related Efforts", p. 20). This concept anchors the theme Framing safety as accidents, not superintelligence alongside Accidents in machine learning systems, the Cleaning robot (running example), and Asimov's first law of robotics, which together show the paper staking out a third position between speculation too remote to test and fiction too informal to specify.