One of the main goals of natural language processing (NLP) is synthetic understanding of natural language documents, especially reading comprehension (RC). An obstacle to the further development of RC systems is the absence of a synthetic methodology to analyze their performance. It is difficult to examine the performance of systems based solely on their results for tasks because the process of natural language understanding is complex. In order to tackle this problem, we propose in this paper a methodology inspired by unit testing in software engineering that enables the examination of RC systems from multiple aspects. Our methodology consists of three steps. First, we define a set of prerequisite skills for RC based on existing NLP tasks. We assume that RC capability can be divided into these skills. Second, we manually annotate a dataset for an RC task with information regarding the skills needed to answer each question. Finally, we analyze the performance of RC systems for each skill based on the annotation. The last two steps highlight two aspects: the characteristics of the dataset, and the weaknesses in and differences among RC systems. We tested the effectiveness of our methodology by annotating the Machine Comprehension Test (MCTest) dataset and analyzing four existing systems (including a neural system) on it. The results of the annotations showed that answering questions requires a combination of skills, and clarified the kinds of capabilities that systems need to understand natural language. We conclude that the set of prerequisite skills we define are promising for the decomposition and analysis of RC.

Artificial intelligence can now outperform humans in a number of advanced tasks, but when it comes to pondering life's greatest mysteries, they're still just as lost as we are. Google researchers have developed a chatbot that can carry out a natural conversation with a human, even demonstrating common sense reasoning. The system can generate solutions in an IT helpdesk scenario and even weigh in on the meaning of life – but with responses like'to live forever' and'to find out what happens when we get to the planet earth,' it doesn't quite have all the answers yet. Google researchers have developed a chatbot that can carry out a natural conversation with a human, even demonstrating common sense reasoning. In one conversation, the human participant asks the machine, 'What is the purpose of life?' A stock image is pictured Machine: to find out what happens when we get to the planet earth .

Artificial Intelligence (which I'll refer to hereafter by its nickname, "AI") is the subfield of Computer Science devoted to developing programs that enable computers to display behavior that can (broadly) be characterized as intelligent.[1] Most research in AI is devoted to fairly narrow applications, such as planning or speech-to-speech translation in limited, well defined task domains. But substantial interest remains in the long-range goal of building generally intelligent, autonomous agents,[2] even if the goal of fully human-like intelligence is elusive and is seldom pursued explicitly and as such. Throughout its relatively short history, AI has been heavily influenced by logical ideas. AI has drawn on many research methodologies: the value and relative importance of logical formalisms is questioned by some leading practitioners, and has been debated in the literature from time to time.[3]

Siri: Okay, from now on I'll call you "an ambulance." Apple fixed this error shortly after its virtual assistant was first released in 2011. But a new contest shows that computers still lack the common sense required to avoid such embarrassing mix-ups. The results of the contest were presented at an academic conference in New York this week, and they provide some measure of how much work needs to be done to make computers truly intelligent. The Winograd Schema Challenge asks computers to make sense of sentences that are ambiguous but usually simple for humans to parse.

In a 1977 article, the late AI pioneer Allen Newell foresaw a time when the entire man-made world would be permeated by systems that cushioned us from dangers and increased our abilities: smart vehicles, roads, bridges, homes, offices, appliances, even clothes. Systems built around AI components will increasingly monitor financial transactions, predict physical phenomena and economic trends, control regional transportation systems, and plan military and industrial operations. Basic research on common sense reasoning, representing knowledge, perception, learning, and planning is advancing rapidly, and will lead to smarter versions of current applications and to entirely new applications. As computers become ever cheaper, smaller, and more powerful, AI capabilities will spread into nearly all industrial, governmental, and consumer applications. Moreover, AI has a long history of producing valuable spin-off technologies.

Endowing computers with common sense is one of the major long-term goals of Artificial Intelligence research. One approach to this problem is to formalize commonsense reasoning using representations based on formal logic or other formal representations. The challenges to creating such a formalization include the accumulation of large amounts of knowledge about our everyday world, the representation of this knowledge in suitable formal languages, the integration of different representations in a coherent way, and the development of reasoning methods that use these representations.

Ever since we were classmates in our AI course (CS 486) at the University of Waterloo, way back in the summer of 2010, our vision has been to solve artificial general intelligence by creating literate machines that could think, reason and communicate like humans. Understanding human language is an extremely complex task and, ultimately, the holy grail in the field of AI. In early 2014, we observed great leaps in the fields of computer vision and speech recognition and pondered the potential of Deep Learning and Reinforcement Learning to enable our mission of creating literate machines. We realized that a great opportunity lay ahead, where machines could learn to model the intelligence and decision-making capabilities of the human brain. This meant more than simple pattern matching on text, but building systems that can actually comprehend, synthesize, infer and make logical decisions like humans. So far, our team has focused on the areas of machine reading comprehension, dialogue understanding, and general (human) intelligence capabilities such as memory, common-sense reasoning, and information seeking behavior.

Microsoft has acquired Canadian startup Maluuba, a company founded by University of Waterloo grads Kaheer Suleman and Sam Pasupalak that also participated in TechCrunch's 2012 San Francisco Startup Battlefield competition. Maluuba focuses on natural language processing, in service of pursuing general artificial intelligence, or building computers that can think like people. The Montreal-based company focuses on using deep learning and reinforcement learning to increase the proficiency and effectiveness of computer-based systems that can answer questions and make decisions, and Microsoft notes in a blog post that its work will help with Microsoft's broad goal of making AI more accessible and useful to the general public. Maluuba's focus has been on improving computer systems' ability to comprehend what they're reading, to understand natural dialog between individuals and to get better at tasks like memory, common-sense reasoning and finding information when they have a gap in their own knowledge. These are huge problems to tackle, and Maluuba notes that it became "apparent" that the best way to make progress was to tap into the significant resources made available from a larger partner.

The dream of thinking machines goes back centuries, at least to Gottfried Wilhelm Leibniz, in the 17th century. Leibniz (right) helped invent mechanical calculators, independently of Isaac Newton developed the integral calculus, and had a lifelong fascination with reducing thinking to calculation. His Mathesis Universalis was a vision of universal science made possible by a mathematical language more precise than natural languages, like English. The Limits of Modern AI: A Story In the 18th Century the Enlightenment philosopher and proto-psychologist Étienne Bonnot de Condillac imagined a statue outwardly appearing like a man and also with what he called "the inward organization." In an example of supreme armchair speculation, Condillac imagined pouring facts--bits of knowledge--into its head, wondering when intelligence would emerge. Condillac's musings drew inspiration from the early mechanical philosophy of Thomas Hobbes, who had famously declared that thinking was nothing but ...