Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Open in new window