KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings

Question answering (QA) has been a longstanding challenge in the field of artificial intelligence. Numerous research works have pushed forward techniques for building QA systems. Many existing approaches achieve high performance on benchmark datasets. However, most of the questions in those datasets only require surface-level reasoning, and do not reveal the full-scale complexity and challenge of the question answering problem. Recently, the AI2 Reasoning Challenge (ARC) has been proposed [Clark et al., 2018], which is designed to pose a challenge to the QA community. On the ARC Challenge Set, several state-of-the-art QA systems, including leading neural models from the well-known SQuAD and SNLI tasks, only perform slightly better than the random baseline. This striking observation has demonstrated that QA is still far from being solved. Why it is so difficult to answer the questions in the ARC Challenge Set? 1) ARC consists of natural science questions, namely questions authored for human exams. All of these questions are drawn from real exams; 2) In order to encourage progress on hard questions, a Challenge Set has been partitioned from ARC.