LM-CORE: Language Models with Contextually Relevant External Knowledge