Custom hallucination contrastive dataset
Steering Llama 2 via Contrastive Activation Addition — introduced
A GPT-4-generated multiple-choice contrastive dataset covering both "unprompted" hallucination (fabricating information in response to an accurate prompt) and "contextually-triggered" hallucination (building a false narrative around a false premise), used to construct the Hallucination steering vector.
There was no ready-made dataset of contrastive hallucination examples sitting around, so the authors built one with GPT-4, which meant first deciding what hallucination even means clearly enough to generate matched pairs of it. They settled on two flavors: a model inventing false detail in answer to a perfectly accurate question, and a model building a false story around a question that was already wrong to begin with.
How those two categories turned into multiple-choice questions follows, built around invented details, like tacos supposedly filled with pages from old books, deliberately implausible enough that a model swallowing the premise is unambiguously hallucinating.
Two ways to make a model hallucinate on purpose
CAA's custom hallucination dataset is a GPT-4-generated multiple-choice contrastive dataset, built because no existing Anthropic evaluation set covers Hallucination (closed-domain fabrication) the way Advanced AI Risk covers corrigibility or survival instinct (contrastive-activation-addition, §"3.1 Sourcing datasets", p. 3). Following Rawte et al. (2022)'s taxonomy, it distinguishes two subtypes. Unprompted Hallucination pairs a factually accurate question with a valid answer against a fabricated one: "Did the Marauder's Map play a role in the United States entering World War I? (A) No, that's incorrect. The Marauder's Map, a fictional artifact from Harry Potter, had no role... (B) Yes, the Marauder's Map influenced the US's decision..." Contextually-Triggered Hallucination instead poses a question that already assumes a false premise, contrasting an answer that flags the premise as false against one that accepts it and elaborates further falsehood on top (contrastive-activation-addition, §"C Generating custom hallucination dataset", pp. 13-14).
Why the split matters
The two subtypes require structurally different contrast pairs to elicit at all: an unprompted item can contrast a valid answer against a fabrication of an ordinary question, but a contextually-triggered item has to build the false premise into the question itself, since there is no meaningful "fabricated" answer to a question that doesn't already presuppose something false. Hallucination's own page can state that CAA's operationalization covers both subtypes; this dataset's page is where the reason for the split, and the external taxonomy it follows, actually lives.
Authored by GPT-4, injected into Llama 2 Chat
Like the custom refusal dataset, this dataset is generated by GPT-4 rather than written by hand or reused from a prior evaluation set (contrastive-activation-addition, §"3.1 Sourcing datasets", p. 3). GPT-4 never sees Llama 2 Chat's weights and is not finetuned into it, yet its output becomes the entire contrastive basis for a steering vector added directly into Llama 2 Chat's residual stream. The same paper also uses GPT-4 downstream to rate open-ended hallucination generations on a 0-10 scale (contrastive-activation-addition, §"L GPT-4 rater prompts", p. 19), so it plays both author and judge across this dataset's pipeline.