Training
Groupwise Advantage Redistribution
2026ResearchUpdated: 22 September 2026Published
Key
innovation
Redistributing advantage within a rollout group instead of independent estimates.
Category
Training
Abstraction level
Building block
Operation level
Post-training
Use cases
LLM RLRL training stabilizationGroup-based RL (GRPO)
How it works
The advantage is recomputed and redistributed within a group of rollouts of the same query to more clearly separate better and worse responses.
Problem solved
Naive advantage estimates within a group can misallocate credit, leading to unstable or weak learning.