GPT-3.5
Sparse Autoencoders Find Highly Interpretable Features in Language Models — inherited
OpenAI's GPT-3.5 model, used as the 'simulator' in the autointerpretability pipeline: given a feature's GPT-4-written explanation, it predicts the feature's per-token activations on held-out text, which are then correlated with ground truth to produce the interpretability score. Substituted for GPT-4 (used for simulation in the original Bills et al. protocol) because OpenAI's public GPT-3.5 API exposed the logprobs the simulation step requires, while its GPT-4 API did not at the time.
A model best known for holding conversations shows up here doing something much narrower: reading a short written explanation of what a feature detects and guessing, token by token, how strongly that feature would fire on a fresh piece of text. It isn't used for anything resembling its usual strengths, and it wasn't even the first choice for the role — an earlier version of this exact procedure used a more capable model instead, until an API limitation forced a swap.
This page covers what the model actually is, why it ends up filling this specific mechanical role instead of the more capable model that writes the explanations in the first place, and how its predictions get turned into a single interpretability score.
A general-purpose model reused for a narrow, mechanical job
GPT-3.5 appears in this paper not as an object of study but as one half of the Autointerpretability score pipeline: the "simulator" that turns a feature's free-text explanation back into a testable prediction. Its entire contribution is mechanical -- read an explanation written by GPT-4, then guess per-token activation numbers -- rather than anything drawing on general conversational ability, which is what makes it usable as a scoring instrument rather than merely a chatbot.
What GPT-3.5 is
GPT-3.5 is OpenAI's model family that succeeded the original GPT-3 (Brown et al., 2020) and underlay the initial public release of ChatGPT in November 2022, later exposed via API as models such as gpt-3.5-turbo. Like GPT-3, it is an autoregressive transformer trained first on next-token prediction and then refined with instruction tuning and reinforcement learning from human feedback, giving it the ability to follow a described task -- such as "predict this feature's activation on each token" -- without task-specific training. The paper treats it as a fixed external tool, taking its capabilities and API behavior as given rather than studying its architecture or training.
Chosen for its logprobs, not for being the better model
GPT-3.5's role here is a direct substitution for a step Bills et al. (2023)'s original autointerpretability protocol assigned to GPT-4 itself. The swap is explained as a practical constraint rather than a methodological preference: "the simulation protocol requires the model's logprobs for scoring, and OpenAI's public API for GPT-3.5 (but not GPT-4) supports returning logprobs" (sparse-autoencoders, §"A AUTOINTERPRETATION PROTOCOL", p. 11). GPT-3.5 therefore never sees the curated top-activating fragments or the numeric activations GPT-4 used to write its explanation -- it works from the explanation's prose alone, predicting per-token activations on 5 highly-activating and 5 randomly sampled fragments, which are then correlated against the feature's true activations to produce the score (sparse-autoencoders, §"A AUTOINTERPRETATION PROTOCOL", p. 12).