Unsupervised value iteration — extracts a complementary signal to → Supervised reward learning

These two proposals mine the same pool of unlabeled episodes for two non-overlapping kinds of information. Supervised reward learning treats the missing reward as the thing to be estimated, training a predictor from state to reward so unlabeled episodes can be scored by proxy. Unsupervised value iteration instead ignores reward entirely and uses the observed state transitions of those same unlabeled episodes to make more accurate Bellman updates (concrete-problems, §"5 Scalable Oversight", p. 12) — information about how the environment moves, independent of what it pays. Because one technique squeezes reward-shaped information out of states and the other squeezes dynamics-shaped information out of transitions, nothing in the paper's framing stops a single semi-supervised RL agent from running both simultaneously on the same batch of unlabeled data.