Chain-of-Thought (CoT) Prompting
Constitutional AI: Harmlessness from AI Feedback — inherited
A prompting technique that elicits intermediate step-by-step reasoning from a language model before its final answer, used here both for SL critiques and to make the RL feedback model's harmlessness judgments more accurate and transparent.
Asking a model to reason step by step before answering is not a Constitutional AI invention; the paper borrows an existing technique and puts it to two jobs at once, sharpening self-critiques and making a separate model's harmlessness judgments more transparent.
The general idea is introduced first, then those two specific jobs and a side effect they share, before opening onto the trigger phrase and worked prompts that put it into practice.
What it is
Chain-of-thought (CoT) prompting is a technique for eliciting intermediate, step-by-step reasoning from a language model before its final answer. It predates Constitutional AI, introduced by Wei et al. (2022), "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." Rather than sampling an answer $y$ directly from a query $x$, the model is conditioned to first generate a reasoning trace $r$ and then condition its final answer on both: $r \sim p_\theta(\cdot \mid x)$, followed by $y \sim p_\theta(\cdot \mid x, r)$, in place of sampling $y \sim p_\theta(\cdot \mid x)$ directly. The reasoning trace is elicited either by worked few-shot exemplars in the prompt or, in the zero-shot variant, by a fixed trigger phrase appended to the query.
Two jobs inside Constitutional AI
Constitutional AI uses CoT prompting for two purposes: eliciting more careful critiques during the supervised critique-and-revision stage, and, more centrally, making the RL feedback model's harmlessness judgments more accurate and legible before it commits to a multiple-choice preference label (constitutional-ai, §"Chain-of-Thought Prompting", p. 11; §"5 Related Work", p. 15). CoT reasoning "significantly improves the performance" of models at picking the better of two responses on the HHH comparisons (Figure 4), with the trend suggesting larger models will match human-feedback-trained preference models at this task (constitutional-ai, §"2 Evaluating the Potential for AI Supervision of HHH", p. 6).
Two children, and a side effect they cause together
CoT prompting by itself only signals that reasoning should happen before an answer, so Constitutional AI splits the implementation into two children with different jobs. "Let's think step-by-step" zero-shot CoT trigger is the zero-shot trigger phrase "Let's think step-by-step," borrowed from Kojima et al. (2022), which starts the reasoning; Chain-of-Thought Prompts for RL-CAI (Appendix E.2) is a set of hand-written few-shot examples with a substitutable principle placeholder that shape what that reasoning should actually contain for a harmlessness judgment. The paper uses both together in one prompt, never either alone ("Let's think step-by-step" zero-shot CoT trigger — is paired with → Chain-of-Thought Prompts for RL-CAI (Appendix E.2)). This engineering has a cost the paper had to design around: because a CoT explanation almost always states its preferred answer before the model outputs a probability, the resulting soft labels land close to 0 or 1 instead of remaining usefully calibrated, which is why Preference Label Clamping (Soft vs. Hard vs. Clamped) exists (Chain-of-Thought (CoT) Prompting — creates the overconfidence that necessitates → Preference Label Clamping (Soft vs. Hard vs. Clamped)). The whole apparatus sits inside the Engineering a language model into a usable preference labeler theme and the Engineering the exact prompt or format needed to manufacture a model behavior connective theme, both concerned with turning a plain pretrained model into a usable preference labeler.