On the Semantics of LM Latent Space: A Vocabulary-defined Approach