Bridging Human and LLM Judgments: Understanding and Narrowing the Gap