What makes your model a low-empathy or warmth person: Exploring the Origins of Personality in LLMs