The First Winograd Schema Challenge at IJCAI-16

Six systems were entered, exploiting a variety of technologies. None of the systems were able to advance from the first round to the second and final round. The Winograd Schema Challenge is concerned with finding the referents of pronouns, or solving the pronoun disambiguation problem. Doing this correctly appears to rely on having a solid base of commonsense knowledge and the ability to reason intelligently with that knowledge. This can be seen from considering an example of a Winograd schema. The referent of it in sentence 1 is the backpack; the referent of it in sentence 2 is the water bottle.

General videogame playing has come a long way in a short period of time, but remains at the level of solving relatively short games made up of distinct and isolated episodes. Even simple console role-playing games (RPGs) are far beyond the reach of current techniques, requiring the synthesis of cultural knowledge with compositional reasoning over several interconnected sub-games. We explore how the challenges of playing these games could spark new advances in compositional analysis of games and common-sense reasoning. General RPG playing can leverage advances in episodic general game playing and in areas like text understanding, image classification, and automated game design learning. It has direct applications in design support and AI-based game design, and the techniques used to enable it could generalize to other families of games such as adventure, open-world, and simulation games. In this paper, we describe the motivation behind general RPG playing in a sub-domain of Nintendo Entertainment System (NES) RPGs, some promising approaches to some of its fundamental issues, and immediate next steps; we conclude by describing a few concrete benchmark problems on the path towards automated play of these complex games.

Commonsense reasoning is in principle a central problem in artificial intelligence, but it is a very difficult one. One approach that has been pursued since the earliest days of the field has been to encode commonsense knowledge as statements in a logic-based representation language and to implement commonsense reasoning as some form of logical inference. This paper surveys the use of logic-based representations of commonsense knowledge in artificial intelligence research.

Hope you like it, I definitely think there should be more interactive demo of NLP systems like this!

We present a framework that learns commonsense temporal knowledge from word definitions. Our work differs from existing systems in both the way definitions are axiomatized and the way knowledge is inferred from those axioms. First, we go beyond axiomatizing just the literal interpretation of a definition by considering the underlying subtext and assumptions a reader has to make to understand a definition. Secondly, we cluster the concept axioms into small event theories that we use to predict the co-occurrence of concepts in simple scenarios. These predictions allow us to identify knowledge derived from the complex interactions among several definitions that would otherwise be ignored. We show that this framework can derive temporal knowledge across several different concept domains. Results are compared to human judgment and demonstrate the effect several features have on evaluation scores.

Machine learning has become pervasive in multiple domains, impacting a wide variety of applications, such as knowledge discovery and data mining, natural language processing, information retrieval, computer vision, social and health informatics, ubiquitous computing, etc. Two essential problems of machine learning are how to generate features and how to acquire labels for machines to learn. Particularly, labeling large amount of data for each domain-specific problem can be very time consuming and costly. It has become a key obstacle in making learning protocols realistic in applications. In this paper, we will discuss how to use the existing general-purpose world knowledge to enhance machine learning processes, by enriching the features or reducing the labeling work. We start from the comparison of world knowledge with domain-specific knowledge, and then introduce three key problems in using world knowledge in learning processes, i.e., explicit and implicit feature representation, inference for knowledge linking and disambiguation, and learning with direct or indirect supervision. Finally we discuss the future directions of this research topic.

In order for AI systems to enhance quality of life, both personally and professionally, they must acquire broad and deep knowledge from multiple domains, learn continuously from interactions with people and environments, and support reasoned decisions. Broadly, the AI fields' long-term progress depend upon many advances. As AI systems become ubiquitous in people's lives, serving many purposes in both personal and professional tasks, there are still many things they cannot do or that they should do much better. In order for AI systems to enhance humans' quality of life, both personally and professionally, they must acquire broad and deep knowledge from multiple domains, learn continuously from interactions with people and environments, and support reasoned decisions. Significant research efforts should be devoted to address these deficiencies.

One of the main goals of natural language processing (NLP) is synthetic understanding of natural language documents, especially reading comprehension (RC). An obstacle to the further development of RC systems is the absence of a synthetic methodology to analyze their performance. It is difficult to examine the performance of systems based solely on their results for tasks because the process of natural language understanding is complex. In order to tackle this problem, we propose in this paper a methodology inspired by unit testing in software engineering that enables the examination of RC systems from multiple aspects. Our methodology consists of three steps. First, we define a set of prerequisite skills for RC based on existing NLP tasks. We assume that RC capability can be divided into these skills. Second, we manually annotate a dataset for an RC task with information regarding the skills needed to answer each question. Finally, we analyze the performance of RC systems for each skill based on the annotation. The last two steps highlight two aspects: the characteristics of the dataset, and the weaknesses in and differences among RC systems. We tested the effectiveness of our methodology by annotating the Machine Comprehension Test (MCTest) dataset and analyzing four existing systems (including a neural system) on it. The results of the annotations showed that answering questions requires a combination of skills, and clarified the kinds of capabilities that systems need to understand natural language. We conclude that the set of prerequisite skills we define are promising for the decomposition and analysis of RC.

Machine learning about language can be improved by supplying it with specific knowledge and sources of external information. We present here a new version of the linked open data resource ConceptNet that is particularly well suited to be used with modern NLP techniques such as word embeddings. ConceptNet is a knowledge graph that connects words and phrases of natural language with labeled edges. Its knowledge is collected from many sources that include expert-created resources, crowd-sourcing, and games with a purpose. It is designed to represent the general knowledge involved in understanding language, improving natural language applications by allowing the application to better understand the meanings behind the words people use. When ConceptNet is combined with word embeddings acquired from distributional semantics (such as word2vec), it provides applications with understanding that they would not acquire from distributional semantics alone, nor from narrower resources such as WordNet or DBPedia. We demonstrate this with state-of-the-art results on intrinsic evaluations of word relatedness that translate into improvements on applications of word vectors, including solving SAT-style analogies.