PPO needs a separate value network to estimate the baseline for its advantage. GRPO samples a group of answers per prompt and uses their mean reward as the baseline, so the advantage is just how a sample compares to its peers, no critic required.
Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
PPO needs a separate value network to estimate the baseline for its advantage. GRPO samples a group of answers per prompt and uses their mean reward as the baseline, so the advantage is just how a sample compares to its peers, no critic required.
The screen is whether the candidate understands that PPO's advantage A = reward minus a learned value baseline, and that GRPO replaces that learned baseline with the empirical mean reward over a group of sampled completions for the same prompt, normalizing within the group. That removes the value model (half the memory and a tuning headache) and works well when rewards are cheap and verifiable (math, code unit tests). The reserved follow-up: 'what does GRPO lose by dropping the critic, and when does the group baseline get noisy.' Saying GRPO is just PPO without a value head, with no mention of the group-relative advantage, is shallow.
No comments yet — be the first to share your approach.
