H-infinity control
Concrete Problems in AI Safety — inherited
A prior robust control-theory framework for minimizing worst-case system response under bounded disturbances, cited as related to bounded exploration.
Control engineers of the early 1980s built a theory around a pessimistic question: how badly can a bounded disturbance shake a system, and how do you design for that worst case? H-infinity control is that framework, and it enters this wiki because AI safety keeps borrowing its way of thinking.
The sections contrast worst-case with average-case design, then trace the safety mechanisms the framework underlies, bounded exploration among them.
Optimizing for the worst case, not the average case
H-infinity control is a framework from robust control theory, developed in the early 1980s, for designing a controller that minimizes a system's worst-case response to a bounded disturbance rather than its average-case response. Given a system with disturbance input $w$ and error output $z$ related through a closed-loop transfer function $T_{zw}$ under some controller $K$, the design objective is to minimize the $H_\infty$ norm $$\|T_{zw}\|_\infty = \sup_{\omega} \, \sigma_{\max}\big(T_{zw}(j\omega)\big),$$ the largest possible gain from disturbance energy to error energy, taken over all frequencies. Minimizing this quantity yields a controller with a guaranteed bound on worst-case deviation, a minimax guarantee, in contrast to $H_2$ or LQG control, which minimizes expected or average-case deviation and offers no such bound against an adversarially chosen disturbance. This concept predates Concrete Problems and is inherited by it as background for bounded exploration (concrete-problems, §"Bounded Exploration:", p. 15).
Two safety mechanisms it underlies
The paper flags Bounded exploration as "related to" H-infinity control without spelling out the analogy (concrete-problems, §"Bounded Exploration:", p. 15); Bounded exploration — borrows its worst case framing from → H-infinity control makes the connection explicit: both substitute a guarantee over a bounded set of possible futures for a guarantee over one actual outcome, trading generality for a hard safety floor. H-infinity control also has a sibling relationship with Reachability analysis, another control-theory import the paper uses for constructing conservative baseline policies; H-infinity control — reaches the same envelope by a different route than → Reachability analysis notes both compute a conservative operating envelope but by different mathematics, a minimax optimization over disturbances for H-infinity control against a set-membership computation of reachable states for reachability analysis. Both concepts belong to Formal verification and control theory recur as safety building blocks, the theme documenting control theory and formal verification as recurring safety infrastructure across the paper's five problem areas.