Beta. Content is under active construction and has not been peer-reviewed. Report errors on
GitHub
.
Disclaimer
Theorem
Path
Curriculum
Paths
Demos
Diagnostic
Search
Quiz Hub
/
Constitutional AI
Constitutional AI
3 questions
Difficulty 5-6
View topic
Intermediate
0 / 3
3 intermediate
Adapts to your performance
1 / 3
intermediate (5/10)
compare
Anthropic's Constitutional AI (Bai et al. 2022) replaces human preference labels with AI-generated critiques. What does it replace in the RLHF pipeline?
Hide and think first
A.
The pretraining corpus, replacing web text with carefully curated 'constitutional' examples
B.
The tokenizer, adding special tokens for 'constitutional' phrases to steer generation
C.
Human preference labels in the reward-model training step, replaced by an AI model critiquing responses against a written 'constitution'
D.
The gradient descent optimizer with a constraint-based solver that respects the constitution
Submit Answer