Provably Mitigating Overoptimization in RLHF: Y our SFT Loss is Implicitly an Adversarial Regularizer

Nov-20-2025, 07:19:10 GMT–Neural Information Processing Systems 

Then it fine-tunes the LLM to maximize the learned reward using RL techniques.