Soft Preference Optimization: Aligning Language Models to Expert Distributions

Open in new window