Constitutional AI: Harmlessness from AI Feedback
Bai, Kadavath, Kundu, et al.
2022 arXiv 2022
TL;DR
Replace most human harmlessness labels with model-generated self-critiques guided by a written set of principles ("constitution"). Scales alignment data and gives more transparent rules.