The Winograd Schema Challenge - a set of twin sentences involving pronoun reference disambiguation that seem to require the use of commonsense knowledge - was proposed by Hector Levesque in 2011. By 2019, a number of AI systems, based on large pre-trained transformer-based language models and fine-tuned on these kinds of problems, achieved better than 90% accuracy. In this paper, we review the history of the Winograd Schema Challenge and discuss the lasting contributions of the flurry of research that has taken place on the WSC in the last decade. We discuss the significance of various datasets developed for WSC, and the research community's deeper understanding of the role of surrogate tasks in assessing the intelligence of an AI system.

The First Winograd Schema Challenge at IJCAI-16

Six systems were entered, exploiting a variety of technologies. None of the systems were able to advance from the first round to the second and final round. The Winograd Schema Challenge is concerned with finding the referents of pronouns, or solving the pronoun disambiguation problem. Doing this correctly appears to rely on having a solid base of commonsense knowledge and the ability to reason intelligently with that knowledge. This can be seen from considering an example of a Winograd schema. The referent of it in sentence 1 is the backpack; the referent of it in sentence 2 is the water bottle.

The Winograd Schema Challenge was proposed by Hector Levesque in 2011 as an alternative to the Turing Test. Chief among its features is a simple question format that can span many commonsense knowledge domains. Questions are chosen so that they do not require specialized knoweldge or training, and are easy for humans to answer. This article details our plans to run the WSC and evaluate results.

Turing test turns out to be highly susceptible to systems that few people would wish to call intelligent. The Loebner Prize Competition (Christian 2011) is in particular associated with the development of chatterbots that are best viewed as successors to ELIZA (Weizenbaum 1966), the program that fooled people into thinking that they were talking to a human psychotherapist by cleverly turning a person's statements into questions of the sort a therapist would ask. The knowledge and inference that characterize conversations of substance -- for example, discussing alternate metaphors in sonnets of Shakespeare -- and which Turing presented as examples of the sorts of conversation that an intelligent system should be able to produce, are absent in these chatterbots. The focus is merely on engaging in surfacelevel conversation that can fool some humans who do not delve too deeply into a conversation, for at least a few minutes, into thinking that they are speaking to another person. The test taker, however, who is given a commonsense knowledge.

This paper describes the Winograd Schema Challenge (WSC), which has been suggested as an alternative to the Turing Test and as a means of measuring progress in commonsense reasoning. A competition based on the WSC has been organized and announced to the AI research community. The WSC is of special interest to the AI applications community and we encourage its members to participate.

John McCarthy's contributions to computer science and artificial intelligence are legendary. He invented Lisp, made substantial contributions to early work in timesharing and the theory of computation, and was one of the founders of artificial intelligence and knowledge representation. This article, written in honor of McCarthy's 80th birthday, presents a brief biography, an overview of the major themes of his research, and a discussion of several of his major papers.

John McCarthy's contributions to computer science and artificial intelligence are legendary. He invented Lisp, made substantial contributions to early work in timesharing and the theory of computation, and was one of the founders of artificial intelligence and knowledge representation. This article, written in honor of McCarthy's 80th birthday, presents a brief biography, an overview of the major themes of his research, and a discussion of several of his major papers.

The 2007 Spring Symposium Series was held Monday through Wednesday, March 26-28, 2007, at Stanford University, California. The titles of the nine symposia in this symposium series were (1) Control Mechanisms for Spatial Knowledge Processing in Cognitive/Intelligent Systems, (2) Game Theoretic and Decision Theoretic Agents, (3) Intentions in Intelligent Systems, (4) Interaction Challenges for Artificial Assistants, (5) Logical Formalizations of Commonsense Reasoning, (6) Machine Reading, (7) Multidisciplinary Collaboration for Socially Assistive Robotics, (8) Quantum Interaction, and (9) Robots and Robot Venues: Resources for AI Education.

The 2007 Spring Symposium Series was held Monday through Wednesday, March 26-28, 2007, at Stanford University, California. The titles of the nine symposia in this symposium series were (1) Control Mechanisms for Spatial Knowledge Processing in Cognitive/Intelligent Systems, (2) Game Theoretic and Decision Theoretic Agents, (3) Intentions in Intelligent Systems, (4) Interaction Challenges for Artificial Assistants, (5) Logical Formalizations of Commonsense Reasoning, (6) Machine Reading, (7) Multidisciplinary Collaboration for Socially Assistive Robotics, (8) Quantum Interaction, and (9) Robots and Robot Venues: Resources for AI Education.