Behavioral clustering — localizes suddenly in → Residual stream

Behavioral clustering does not fade in gradually across a model's depth -- it appears abruptly at a specific residual-stream layer. For the refusal dataset in Llama 2 7B Chat, PCA projections at layer 9 show no separation by behavior, while the same projection one layer later, at layer 10, shows a clean split: "the visible behavioral clustering emerges suddenly at layer 10. This trend is seen across our other datasets" (contrastive-activation-addition, §"3.2 Visualizing activations for contrastive dataset analysis", p. 4). Across the seven behaviors tested, this onset consistently falls around one-third of the way through the layers, which the authors link to prior work showing emotion representations likewise emerge in middle and later layers rather than early ones. The residual stream is therefore not a uniform medium in which behavioral information accumulates smoothly from layer one -- it is largely inert to a behavior's presence for the model's first third, then becomes linearly readable within a single layer, which is also why CAA's own layer sweeps concentrate their steering effect there rather than earlier.