GPT-4 — generates explanations for → Autointerpretability score

Of the 20 top-activating fragments identified per feature from the first 50,000 lines of OpenWebText, GPT-4 is shown only 5, together with each fragment's per-token activations rescaled to integers 0-10, and from that narrow window writes the single free-text explanation that the rest of the pipeline treats as ground truth. Because GPT-3.5's later simulation step never sees the original fragments or activation numbers, GPT-4's explanation is the sole channel through which everything the feature 'means' has to pass, so any wording problem in that explanation propagates undetected into the final score. Bills et al. (2023), whose protocol this paper follows, already found this explainer role limited: it struggles to describe patterns keyed to the previous or next token rather than the current one, and has no way to check a candidate explanation against how ablating the feature changes the model's output. Table 1's arbitrary first-five features from the layer-1 dictionary show what this produces in practice: GPT-4 writes a description as concrete as 'the number 5, and also records moderate to low activation for personal names and some nouns' (scored 0.57) alongside one as vague as 'actions performed by a subject or object' (scored -0.11), a negative score meaning the explanation actively anti-predicts the feature's real behavior.