GVPO: Group Variance Policy Optimization for Large Language Model Post-Training

Post-training plays a crucial role in refining and aligning large language models to meet specific tasks and human preferences. While recent advancements in posttraining techniques, such as Group Relative Policy Optimization (GRPO), leverage increased sampling with relative reward scoring to achieve superior performance, these methods often suffer from training instability that limits their practical adoption.