Sharpening Tax in Post-Training

Sharpening Tax studies how post-training changes an agent's success distribution. A tuned model may solve more tasks on its first attempt while repeatedly failing a subset that a less concentrated base policy occasionally solves. That distinction matters when a system can generate several candidates and reliably recognize a successful one.

The authors compare base and post-trained model pairs on interactive benchmarks. Base models receive a lightweight harness that helps them use the required interfaces. The study reports frequent tradeoffs between single-attempt accuracy and success after many independent rollouts, then introduces a statistic summarizing the change in test-time scalability.

The paper also proposes posterior-tempered group sampling, or PTGS. During RL training, it estimates prompt difficulty and varies sampling temperature: harder prompts receive more exploration, while easier prompts receive more concentrated sampling. The reported controlled experiments improve both accuracy and coverage relative to their fixed-temperature baselines.

The metric needs careful interpretation. A positive tax can arise because a policy loses recoverable tasks, but it can also arise because it succeeds sooner and therefore benefits less from retries. Read the tax beside absolute success, consistency and cost. The official project and code provide the study's implementation and experimental context.

A one-shot support assistant and an offline coding search system need different success profiles. The assistant may value a strong first answer and predictable behavior. The coding system may afford several isolated attempts, provided tests can identify a useful result. A single leaderboard score cannot represent both requirements.

The paper provides vocabulary for that decision, but repeated attempts are not free. They consume tokens, latency and evaluator capacity. For agents that mutate state, retries may also require a clean environment or explicit recovery semantics; blindly repeating an irreversible action is not equivalent to independent sampling.

Use inference-time compute to define the real budget and trajectory evaluation to define success. If the system cannot distinguish a good candidate from a bad one, oracle pass@k coverage overstates the result a user will receive.