Counterfactual reasoning
Concrete Problems in AI Safety — inherited
A prior statistical/causal framework (Neyman, Rubin, Pearl) for reasoning about 'what would have happened had the world been different,' proposed as a unifying lens on distributional shift.
What would have happened had the world been different? Statisticians and causal theorists from Neyman to Pearl built formal machinery for answering that question, and counterfactual reasoning is the umbrella this corpus gives it. Concrete Problems in AI Safety (2016) suggests it as a unifying way to think about distributional shift.
One section introduces the causal frameworks themselves; the next applies them as a lens on distributional shift.
What would have happened had the world been different
Counterfactual reasoning is a family of statistical and causal frameworks for answering questions of the form "what would have happened to this unit had it received a different treatment or been drawn from a different distribution?" It long predates Concrete Problems, tracing to Jerzy Neyman's 1923 work on randomized experiments and developed into the modern potential-outcomes framework by Donald Rubin in the 1970s: for a unit $i$ under treatment assignment $T_i \in \{0,1\}$, one posits two potential outcomes $Y_i(1)$ and $Y_i(0)$, only one of which is ever observed, and defines the individual causal effect as $\tau_i = Y_i(1) - Y_i(0)$ — a quantity that is by construction never directly measurable, since a unit cannot simultaneously be observed under both conditions. Judea Pearl's structural causal models supply a graphical and algebraic complement to this framework, formalizing an intervention as the do-operator: $P(Y \mid \mathrm{do}(X = x))$ denotes the distribution of $Y$ that would result from forcibly setting $X = x$, as opposed to the merely observational $P(Y \mid X = x)$, which can differ sharply whenever $X$ and $Y$ share a confounder. Concrete Problems cites this literature (Neyman, Rubin, Pearl) as one of "two viewpoints" the authors find useful for thinking about out-of-distribution prediction (concrete-problems, §"A unifying view: counterfactual reasoning and machine learning with contracts.", p. 19).
As a lens on distributional shift
The paper's proposal is that Robustness to distributional shift can itself be understood as a special case of the counterfactual question: a model trained on distribution $p_0$ and deployed on $p^*$ is being asked, in effect, "what would this model's predictions have been, had the inputs come from a different world?" That reframing is descriptive rather than prescriptive — it explains what distributional shift fundamentally is without yet saying what a system should be obligated to do about it. The edge Counterfactual reasoning — is the descriptive counterpart to → Machine learning with contracts pairs it directly against Machine learning with contracts, the paper's complementary, prescriptive viewpoint on the same problem, and situates both inside the Reframing distributional shift as broken contracts and causal structure theme and the Before the fact or after: when a safety property gets enforced connective theme. Concrete Problems is candid that this connection is more a research direction than a solved technique: it is "excited by recent work applying counterfactual reasoning techniques to machine learning problems," while noting "there appears to be much work remaining to be done to scale these to high-dimensional and highly complex settings" (concrete-problems, §"A unifying view: counterfactual reasoning and machine learning with contracts.", p. 19). The concept sits under Robustness to distributional shift as its parent and belongs to the Distributional shift: relax the statistics, or rewrite the contract supertheme, contributing a causal vocabulary rather than a statistical relaxation to that broader response.