Environment Modifications for Preference Learning — exempts pong from → Synthetic Oracle Feedback

The paper's general rule for Atari strips life-loss and episode-end signals from the agent entirely, converting every game into one continuous episode so that termination itself cannot leak task information (deep-rl-human-prefs, §"A Experimental Details", p. 14). That rule is not applied uniformly once the feedback source is the synthetic oracle: the same passage states that under synthetic oracle feedback, episode ends are replaced with a learned penalty 'in all games except Pong,' leaving Pong as the one environment where the oracle-trained agent still confronts the raw, unmodified episode boundary (deep-rl-human-prefs, §"A Experimental Details", p. 14). The exception is easy to miss because it applies only to the oracle condition, meaning the environment the synthetic oracle is evaluated in is not quite the same environment a human rater's judgments are trained against for that one game. It is a small but concrete illustration of how much bespoke, game-by-game tuning the 'clean' experimental design behind the quantitative comparisons actually required.