Trajectory Segment — generalizes into → RM (comparison) dataset

The atomic unit a human judges in the 2017 paper is a trajectory segment: a clip of one to two seconds of observations and actions, chosen specifically because it is faster to render and evaluate than a whole episode while still carrying enough information to be informative (deep-rl-human-prefs, §"2.1 Setting and Goal", p. 3). InstructGPT's RM dataset judges a different kind of atom entirely — a complete model output, a full block of generated text answering one prompt, with no notion of a partial clip of behavior at all (instructgpt, §"3.2 Dataset", p. 7). What survives the jump from robotics to language is not the unit itself but the design principle behind choosing it: pick the smallest thing a human can evaluate quickly without losing the information the comparison is meant to carry, whatever 'smallest' means in that domain. For a video of continuous motor behavior that meant clipping to a couple of seconds; for language it means comparing whole completions rather than, say, individual tokens, since a token in isolation carries almost none of the information a full response does.