Can Language Model Understand Word Semantics as A Chatbot? An Empirical Study of Language Model Internal External Mismatch