Constitutional AI's method: a written constitution, AI feedback, and its place among prior agents
Ordinary RLHF needs tens of thousands of human harm judgments; Constitutional AI trades nearly all of them for a short list of written principles and lets the model critique and revise its own bad answers instead. What is easy to undersell is how much engineering that trade actually required.
Critique-and-revision opens the method, then the page turns to the narrower engineering problem of making a plain language model's judgments trustworthy enough to train on, and to the red-teamed conversations that feed both stages. It closes by placing the method next to its predecessors, LaMDA and Sparrow included, each borrowed from differently.
Constitutional AI's method: principles, critique, and revision lays out CAI's supervised stage, a small set of written principles standing in for human harm labels, used to critique and revise a model's own harmful responses. Engineering a language model into a usable preference labeler covers the RL stage's counterpart problem, the prompting, chain-of-thought reasoning, and calibration-dependent label clamping needed to make a feedback model's judgments usable as training signal, sharing principle-ensembling with the supervised stage and leaning on model calibration, the same enabling assumption underlying the corpus's shared HHH benchmark. Sourcing and classifying harmful prompts supplies the harmful training data, via crowdworker red-teaming, that both stages consume. Prior dialogue-agent systems Constitutional AI positions itself against, LaMDA, Sparrow, context distillation, locates the method relative to contemporaneous work, with Sparrow's rule-based harmlessness decomposition the closest analogue to CAI's own constitution. Together these themes describe one method, its engineering, its data supply, and its place in the literature.