Reframing distributional shift as broken contracts and causal structure
Distributional shift might not be a statistics problem at all. This theme covers two reframings Concrete Problems (2016) offers instead: a causal one, treating shift through the lens of counterfactual reasoning, and a software-engineering one, treating the match between training and test data as an implicit contract that quietly gets broken.
Both readings develop side by side before the page turns to model repair, the concrete technique that follows from the contract framing: fixing a model after deployment once a promised safety property has failed.
Robustness to distributional shift asks an ML system to perform reasonably, and to know when it is performing badly, once deployed on a test distribution unlike its training one. This theme covers two conceptual reframings the paper offers, distinct from the purely statistical relaxations elsewhere in the corpus. Counterfactual reasoning, a prior causal framework associated with Neyman, Rubin, and Pearl for reasoning about what would have happened had the world been different, is proposed as a unifying lens on what distributional shift fundamentally is. Machine learning with contracts borrows a software-engineering analogy instead, arguing that systems should be built to satisfy explicit, well-defined behavioral contracts rather than relying on the brittle implicit contract that training and test distributions are identical. Model repair supplies a concrete technique within that frame, altering an already-trained model after the fact to ensure a specified safety property holds even when the original contract was violated. Both reframings share a claim: the problem is not just statistical mismatch but a failure of specification.