Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge