One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spaces