Helpful, honest, harmless (HHH) alignment framework
Training language models to follow instructions with human feedback — inherited; Constitutional AI: Harmlessness from AI Feedback — inherited; Steering Llama 2 via Contrastive Activation Addition — inherited; Persona Vectors: Monitoring and Controlling Character Traits in Language Models — inherited
A framework, adapted from Askell et al. (2021), that defines an aligned model as one that is helpful (follows user intent), honest (does not fabricate or mislead), and harmless (avoids causing harm).
InstructGPT and Constitutional AI never stop to argue over what an aligned model should look like, because both borrow the same answer from earlier work: helpful, honest, and harmless. That shared vocabulary is what lets training methods built years apart, including a 2023 steering paper that names the same framework as its own goal, be judged on the same terms.
From there the page follows that borrowed definition through InstructGPT and Constitutional AI before turning to how it gets put into practice and measured, landing on the evaluation set built to score against it.
How it fits — a shared definition across the corpus
The helpful, honest, harmless (HHH) framework is the single vocabulary both InstructGPT and Constitutional AI reach for to state what an aligned model actually is. It is inherited by both papers from Askell et al. (2021) rather than invented by either: InstructGPT glosses its founding objective, "following instructions helpfully and safely," in exactly these terms in its very first section, stating that alignment "encompasses both explicit intentions such as following instructions and implicit intentions such as staying truthful, and not being biased, toxic, or otherwise harmful," and, "using the language of Askell et al. (2021)," that the model should be helpful, honest, and harmless (instructgpt, §"1 Introduction", p. 2; Instruction following as an alternative to next-token prediction — adopts the vocabulary of → Helpful, honest, harmless: a shared definition of alignment). So Instruction following (as a fine-tuning objective) was never literal command compliance that HHH got layered onto afterward — it was defined as the helpful third of the HHH triad from its first statement in the paper. Constitutional AI inherits the identical framework as "the three-part objective for AI safety" it aims to satisfy without human harmfulness labels (constitutional-ai, §"1 Introduction", p. 2; §"6.1 Future Directions", p. 15), though there it narrows: only two of the three terms, helpfulness and harmlessness, enter Constitutional AI's documented empirical tradeoff, with honesty dropping out of the measured tension entirely. Steering Llama 2 via Contrastive Activation Addition (2023) reaches for the identical formula one paper later and one method further removed from training-time intervention: its introduction states that the growing body of alignment research "aims to ensure [LLMs] are 'helpful, honest, and harmless' (Askell et al., 2021)" (contrastive-activation-addition, §"1 Introduction", p. 2), naming CAA's own inference-time steering as another attempt at the same three-part goal InstructGPT and Constitutional AI pursue through training. The framework's reach across this corpus is now a three-paper pattern rather than a two-paper coincidence: InstructGPT's supervised and RL finetuning, Constitutional AI's RL from AI feedback, and CAA's post-hoc activation steering all state their objective in Askell et al.'s own words, even though only the first two are measured against the HHH evaluation set this framework was operationalized as.
What it enabled and how it is operationalized
A qualitative triad needs a measurable instrument, and its child concept, the HHH evaluation set, is exactly that: the framework is operationalized as this benchmark, a set of binary comparisons testing whether a model can identify the more helpful, honest, and harmless of two responses, originally 221 items from Askell et al. (2021) hosted on BIG-Bench, later extended by Constitutional AI with 217 harder harmlessness-focused items once models exceeded 90% accuracy on the original set (constitutional-ai, §"2 Evaluating the Potential for AI Supervision of HHH", p. 6). Notably, the two papers operationalize the "honest" leg differently: InstructGPT measures it via TruthfulQA and hallucination rate rather than via this benchmark at all (instructgpt, §"3.6 Evaluation", p. 9), since a model's actual internal beliefs cannot be directly inspected, so truthfulness stands in as the measurable proxy for honesty.