Learning Loss Landscapes in Preference Optimization