PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs