Core

Policy Optimization: PPO and GRPO

PPO and GRPO are the reinforcement-learning optimizers that turn a reward signal into model weight updates during RLHF and reasoning training. PPO uses a clipped objective and a KL anchor to keep updates stable and close to the reference model; GRPO drops the learned value network and instead normalizes rewards across a group of sampled answers. FDE loops probe these because the KL anchor, reward hacking, and GRPO's cost savings are where reasoning pipelines actually succeed or break.

a free account unlocks the core curriculum tier · no card

RELATED CONCEPTS

RLHF (Alignment)Reward Models Direct Preference Optimization (DPO)

PRACTICE THIS IN REAL QUESTIONS

LLM & GenAI FundamentalsWhat's the difference between a base model and an instruct model, and why would a customer care?→LLM & GenAI FundamentalsDesign the pre-launch eval suite for a customer support bot.→AI Security, Privacy & GovernanceReproduce-from-paper: design a production-safe Constitutional-AI-style fine-tuning pipeline that aligns a chatbot to a set of rules→LLM & GenAI FundamentalsWalk through the RLHF pipeline (SFT, reward model, PPO) and explain how DPO simplifies it.→LLM & GenAI FundamentalsGRPO vs PPO-based RLHF: how does group-relative reward let you drop the value model?→LLM & GenAI FundamentalsTrain a reward model for a coding agent from 100K noisy human scores biased toward short solutions.→