Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data

Open in new window