Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing