Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models