Scalable oversight — is addressed by → Hierarchical reinforcement learning (HRL)

Hierarchical RL addresses scalable oversight through a specific mechanical trick: pushing the expensive, sparse true-objective evaluation up to the top level of a delegation hierarchy while everything below runs on cheap, dense, synthetic reward. The top-level agent takes only a small number of highly abstract, long-timescale actions, so it can tolerate very infrequent access to the real objective; the sub-agents it delegates to, in turn, never need that expensive signal at all, since they are scored on synthetic rewards representing correct completion of the goal they were assigned (concrete-problems, §"5 Scalable Oversight", p. 13). The paper calls this a 'particularly promising' route to oversight precisely because the scarce resource — calls to the true objective — is spent only where the decision space is smallest.