The 2017 deep-RL testbeds: MuJoCo and Atari — repurposes environments for → Novel behaviors without reward functions

The novel-behavior demonstrations are not new environments; they are the same three testbeds run again toward a different, in one case wholly absent, target, a continuity neither theme's own narrative makes explicit. Hopper, matched against a withheld true reward in the benchmark runs, is retrained toward a sequence of backflips that has no true reward function at all; Half-Cheetah, benchmarked on forward speed, is retrained to run balanced on one leg; and Enduro, benchmarked on the game's own passing-cars score, is retrained toward staying level with traffic instead (deep-rl-human-prefs, §"3.2 Novel behaviors", p. 8-9). The paper is explicit that this is the same machinery, unmodified: "Using the same parameters as in the previous experiments, we show that our algorithm can learn novel complex behaviors" (deep-rl-human-prefs, §"3.2 Novel behaviors", p. 8). That continuity is the point of reusing these particular environments rather than fresh ones: the testbeds theme establishes that the method can match a known answer, so when the identical code is pointed at a task with no known answer to match, the results can be read as evidence about the method rather than about some unrelated change in setup, one theme proves competence on a checkable case, the other spends that same competence on tasks where competence can no longer be checked against a number.