Question Answering
Leveraging Wikipedia Characteristics for Search and Candidate Generation in Question Answering
Chu-Carroll, Jennifer (IBM T. J. Watson Research Center) | Fan, James (IBM T. J. Watson Research Center)
Most existing Question Answering (QA) systems adopt a type-and-generate approach to candidate generation that relies on a pre-defined domain ontology. This paper describes a type independent search and candidate generation paradigm for QA that leverages Wikipedia characteristics. This approach is particularly useful for adapting QA systems to domains where reliable answer type identification and type-based answer extraction are not available. We present a three-pronged search approach motivated by relations an answer-justifying title-oriented document may have with the question/answer pair. We further show how Wikipedia metadata such as anchor texts and redirects can be utilized to effectively extract candidate answers from search results without a type ontology. Our experimental results show that our strategies obtained high binary recall in both search and candidate generation on TREC questions, a domain that has mature answer type extraction technology, as well as on Jeopardy! questions, a domain without such technology. Our high-recall search and candidate generation approach has also led to high overall QA performance in Watson, our end-to-end system.
A Natural Language Question Answering System as a Participant in Human Q&A Portals
Dong, Tiansi (University of Hagen) | Furbach, Ulrich (University Koblenz-Landau) | Glöckner, Ingo (University of Hagen) | Pelzer, Björn (University Koblenz-Landau)
LogAnswer is a question answering (QA) system for the German language, aimed at providing concise and correct answers to arbitrary questions. For this purpose LogAnswer is designed as an embedded artificial intelligence system which integrates methods from several fields of AI, namely natural language processing, machine learning, knowledge representation and automated theorem proving. We intend to employ LogAnswer as a virtual user within Internet-based QA forums, where it must be able to identify the questions that it cannot answer correctly, a task that normally receives little attention in QA research compared to the actual answer derivation. The paper presents a machine learning solution to the wrong answer avoidance (WAA) problem, applying a meta classifier to the output of simple term-based classifiers and a rich set of other WAA features. Experiments with a large set of real-world questions from a QA forum show that the proposed method significantly improves the WAA characteristics of our system.
What to Ask to an Incomplete Semantic Web Reasoner?
Grau, Bernardo Cuenca (Oxford University) | Stoilos, Giorgos (Oxford University)
Largely motivated by Semantic Web applications, many highly scalable, but incomplete, query answering systems have been recently developed. Evaluating the scalability-completeness trade-off exhibited by such systems is an important requirement for many applications. In this paper, we address the problem of formally comparing complete and incomplete systems given an ontology schema (or TBox) T. We formulate precise conditions on TBoxes T expressed in the EL, QL or RL profile of OWL 2 under which an incomplete system is indistinguishable from a complete one w.r.t. T, regardless of the input query and data. Our results also allow us to quantify the "degree of incompleteness" of a given system w.r.t. T as well as to automatically identify concrete queries and data patterns for which the incomplete system will miss answers.
Interfacing Virtual Agents With Collaborative Knowledge: Open Domain Question Answering Using Wikipedia-Based Topic Models
Waltinger, Ulli (University Bielefeld) | Breuing, Alexa (University Bielefeld) | Wachsmuth, Ipke (University Bielefeld)
This paper is concerned with the use of conversational agents as an interaction paradigm for accessing open domain encyclopedic knowledge by means of Wikipedia. More precisely, we describe a dialogue-based question answering system for German which utilizes Wikipedia-based topic models as a reference point for context detection and answer prediction. We investigate two different per- spectives to the task of interfacing virtual agents with collaborative knowledge. First, we exploit the use of Wikipedia categories as a basis for identifying the broader topic of a spoken utterance. Second, we describe how to enhance the conversational behavior of the virtual agent by means of a Wikipedia-based question answering component which incorporates the question topic. At large, our approach identifies topic-related focus terms of a user’s question, which are subsequently mapped onto a category taxonomy. Thus, we utilize the taxonomy as a reference point to derive topic labels for a user’s question. The employed topic model is thereby based on explicitly given concepts as represented by the document and category structure of the Wikipedia knowledge base. Identified topic categories are subsequently combined with different linguistic filtering methods to improve answer candidate retrieval and reranking. Results show that the topic model approach contributes to an enhancement of the conversational behavior of virtual agents.
An Approach to Answer Selection in Question-Answering Based on Semantic Relations
Mendes, Ana Cristina (Instituto Superior Técnico, Technical University of Lisbon and Spoken Language Systems Lab/INESC-ID Lisboa) | Coheur, Luísa (Instituto Superior Técnico, Technical University of Lisbon and Spoken Language Systems Lab/INESC-ID Lisboa)
A usual strategy to select the final answer in factoid Question-Answering (QA) relies on redundancy. A score is given to each candidate answer as a function of its frequency of occurrence, and the final answer is selected from the set of candidates sorted in decreasing order of score. For that purpose, systems often try to group together semantically equivalent answers. However, they hold several other semantic relations, such as inclusion, which are not considered, and candidates are mostly seen independently, as competitors. Our hypothesis is that not just equivalence, but other relations between candidate answers have impact on the performance of a redundancy-based QA system. In this paper, we describe experimental studies to back up this hypothesis. Our findings show that, with relatively simple techniques to recognize relations, systems' accuracy can be improved for answers of categories Number, Date and Entity.
Query Answering in the Horn Fragments of the Description Logics SHOIQ and SROIQ
Ortiz, Magdalena (Vienna University of Technology) | Rudolph, Sebastian (Karlsruhe Institute of Technology) | Simkus, Mantas (Vienna University of Technology)
As more and more application areas require higher scalability, the study of fragments of expressive The high computational complexity of the expressive DLs with better computational properties has become an Description Logics (DLs) that underlie the important area of research. OWL standard has motivated the study of their Horn fragments of DLs, which are obtained by restricting Horn fragments, which are usually tractable in data the syntax of a DL in such a way that disjunction is not complexity and can also have lower combined complexity, expressible, were first considered in [Hustadt et al., 2005] particularly for query answering. In this paper as expressive fragments with tractable data complexity (see we provide algorithms for answering conjunctive also [Krötzsch et al., 2007]). It was later identified that they 2-way regular path queries (2CRPQs), a nontrivial can also exhibit lower combined complexity when it comes generalization of plain conjunctive queries, to query answering. Indeed, answering conjunctive queries in the Horn fragments of the DLs SHOIQ and (CQs), a kind of database-inspired queries that have become SROIQ underlying OWL 1 and OWL 2. We show the standard for querying DLs, is in ExpTime for the Horn that the combined complexity of the problem is ExpTime-complete fragment of the prominent SHIQ [Eiter et al., 2008], while it for Horn-SHOIQ and 2ExpTimecomplete is already 2ExpTime-hard for quite restricted (non Horn) fragments for the more expressive Horn-SROIQ, of SHIQ, like ALCI [Lutz, 2008] and SH [Eiter et but is PTime-complete in data complexity for both.
Evaluating Conversational Characters Created through Question Generation
Chen, Grace (California State University Long Beach) | Tosch, Emma (Brandeis University) | Artstein, Ron (USC Institute for Creative Technologies) | Leuski, Anton ( USC Institute for Creative Technologies ) | Traum, David ( USC Institute for Creative Technologies )
Question generation tools can be used to extract a question-answer database from text articles. We investigate how suitable this technique is for giving domain-specific knowledge to conversational characters. We tested these characters by collecting questions and answers from naive participants, running the questions through the character, and comparing the system responses to the participant answers. Characters gave a full or partial answer to 53% of the user questions which had an answer available in the source text, and 43% of all questions asked. Performance was better for questions asked after the user had read the source text, and also varied by question type: the best results were answers to who questions, while answers to yes/no questions were among the poorer performers. The results show that question generation is a promising method for creating a question answering conversational character from an existing text.
True Knowledge: Open-Domain Question Answering Using Structured Knowledge and Inference
Tunstall-Pedoe, William (True Knowledge Ltd)
This article gives a detailed description of True Knowledge: a commercial, open-domain question answering platform. The system combines a large and growing structured knowledge base of common sense, factual and lexical knowledge; a natural language translation system that turns user questions into internal language-independent queries and an inference system that can answer those queries using both directly represented and inferred knowledge. The system is live and answers millions of questions per month asked by internet users.
Project Halo Update--Progress Toward Digital Aristotle
Gunning, David (Vulcan, Inc.) | Chaudhri, Vinay K. (SRI International) | Clark, Peter E. (Boeing Research and Technology) | Barker, Ken (University of Texas at Austin) | Chaw, Shaw-Yi (University of Texas at Austin) | Greaves, Mark (Vulcan, Inc.) | Grosof, Benjamin (Vulcan, Inc.) | Leung, Alice (Raytheon BBN Technologies Corporation) | McDonald, David D. (Raytheon BBN Technologies Corporation) | Mishra, Sunil (SRI International) | Pacheco, John (SRI International) | Porter, Bruce (University of Texas at Austin) | Spaulding, Aaron (SRI International) | Tecuci, Dan (University of Texas at Austin) | Tien, Jing (SRI International)
In the winter, 2004 issue of AI Magazine, we reported Vulcan Inc.'s first step toward creating a question-answering system called "Digital Aristotle." The goal of that first step was to assess the state of the art in applied Knowledge Representation and Reasoning (KRR) by asking AI experts to represent 70 pages from the advanced placement (AP) chemistry syllabus and to deliver knowledge-based systems capable of answering questions from that syllabus. This paper reports the next step toward realizing a Digital Aristotle: we present the design and evaluation results for a system called AURA, which enables domain experts in physics, chemistry, and biology to author a knowledge base and that then allows a different set of users to ask novel questions against that knowledge base. These results represent a substantial advance over what we reported in 2004, both in the breadth of covered subjects and in the provision of sophisticated technologies in knowledge representation and reasoning, natural language processing, and question answering to domain experts and novice users.