Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality