Knowledge Graphs & NLP @ EMNLP 2019 Part I

Language models (LMs) are the hottest topic in the NLP research right now. The most prominent examples are BERT and GPT-2 but new LMs are published every month trained on humongous volumes of text. Are LMs capable of encoding knowledge in a way similar to knowledge graphs? Petroni et al study this problem comparing language models with knowledge graphs on Question Answering and NLG tasks where factual knowledge is required, e.g., a question is posed by inserting a MASK token instead of an answer. Turns out LMs demonstrate similar to KGs performance on very simple questions such as "Adolphe Adam died in [Paris]" .