reinforcement learning

PPO needs a separate value network to estimate the baseline for its advantage. GRPO samples a group of answers per prompt and uses their mean reward as the baseline, so the advantage is just how a sample compares to its peers, no critic required.

The thing interviewers actually screen for isn't the Bellman equation, it's whether you know when NOT to use RL. Most problems that look like RL are better solved by supervised learning or a bandit, and reaching for full RL when you have labels is a classic over-engineering tell.

A red-team model that learns over many episodes which attacks bypass your filters is only useful if it explores instead of farming one exploit, and only safe if every discovered attack becomes target training data.