A Systematic Examination of Preference Learning through the Lens of Instruction-Following