Reproduce-from-paper: design a production-safe Constitutional-AI-style fine-tuning pipeline that aligns a chatbot to a set of rules

The screen is whether the candidate knows the actual CAI mechanism (self-critique and revision for SFT, then a preference model trained on AI-labeled comparisons for the RL phase) and does not confuse it with vanilla RLHF that needs human harm labels. The production senior move is the validation phase: a held-out adversarial suite, per-rule pass rates, and a regression gate, because a model that follows the constitution on the train distribution but folds under a jailbreak is not production-safe. Watch for the candidate who describes RLHF and calls it Constitutional AI.