Goto

Collaborating Authors

Ram, Ashwin


On Evaluating and Comparing Open Domain Dialog Systems

arXiv.org Artificial Intelligence

Conversational agents are exploding in popularity. However, much work remains in the area of non goal-oriented conversations, despite significant growth in research interest over recent years. To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5-million dollar university competition where sixteen selected university teams built conversational agents to deliver the best social conversational experience. Alexa Prize provided the academic community with the unique opportunity to perform research with a live system used by millions of users. The subjectivity associated with evaluating conversations is key element underlying the challenge of building non-goal oriented dialogue systems. In this paper, we propose a comprehensive evaluation strategy with multiple metrics designed to reduce subjectivity by selecting metrics which correlate well with human judgement. The proposed metrics provide granular analysis of the conversational agents, which is not captured in human ratings. We show that these metrics can be used as a reasonable proxy for human judgment. We provide a mechanism to unify the metrics for selecting the top performing agents, which has also been applied throughout the Alexa Prize competition. To our knowledge, to date it is the largest setting for evaluating agents with millions of conversations and hundreds of thousands of ratings from users. We believe that this work is a step towards an automatic evaluation process for conversational AIs.


Robust and Authorable Multiplayer Storytelling Experiences

AAAI Conferences

Interactive narrative systems attempt to tell stories to players capable of changing the direction and/or outcome of the story. Despite the growing importance of multiplayer social experiences in games, little research has focused on multiplayer interactive narrative experiences. We performed a preliminary study to determine how human directors design and execute multiplayer interactive story experiences in online and real world environments. Based on our observations, we developed the Multiplayer Storytelling Engine that manages a story world at the individual and group levels. Our flexible story representation enables human authors to naturally model multiplayer narrative experiences. An intelligent execution algorithm detects when the author's story representation fails to account for player behaviors and automatically generates a branch to restore the story to the authors' original intent, thus balancing authorability against robust multiplayer execution.


Learning Opponent Strategies through First Order Induction

AAAI Conferences

In a competitive game it is important to identify the opponent's strategy as quickly and accurately as possible so that an effective response can be planned. In this vein, this paper summarizes our work in exploring using first order inductive learning to learn rules for representing opponent strategies. Specifically, we use these learned rules to perform plan recognition and classify an opponent strategy as one of multiple learned strategies. Our experiments validate this novel approach in a simple real-time strategy game.


Socio-Semantic Health Information Access

AAAI Conferences

We describe Cobot, a mixed initiative socio-semantic conversational search and recommendation system for finding health information. With Cobot, users can start a real time conversation about their health concerns. Cobot then connects relevant users together in the conversation also providing contextual recommendations relevant to the conversation. Conventional search engines and content portals provide a solitary search experience inundating the health information seeker with a hoard of information often confusing and frustrating them. Cobot brings relevant healthcare information directly or through other users without any search through natural language conversation.


CBArch: A Case-Based Reasoning Framework for Conceptual Design of Commercial Buildings

AAAI Conferences

The paper describes the first phase of development of a Case-Base Reasoning (CBR) system to support early conceptual design of buildings. As specific context of application, the research focuses on energy performance of commercial buildings, and the early identification of energy-related features that contribute to its outcomes. The hypothesis is that bringing knowledge from relevant precedents may facilitate this identification process, thus offering a significant contribution for early analysis and decision-making. The paper introduces a proof-of-concept for such a system, proposing a novel integration of Case-Based Reasoning, Parametric Modeling (Building Information Modeling), and Ontology Classification. Potential advantages and limitations of this three-level integration approach are discussed along with recommendations for future development.


Reports of the AAAI 2010 Conference Workshops

AI Magazine

The AAAI-10 Workshop program was held Sunday and Monday, July 11–12, 2010 at the Westin Peachtree Plaza in Atlanta, Georgia. The AAAI-10 workshop program included 13 workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Fun, Bridging the Gap between Task and Motion Planning, Collaboratively-Built Knowledge Sources and Artificial Intelligence, Goal-Directed Autonomy, Intelligent Security, Interactive Decision Theory and Game Theory, Metacognition for Robust Social Systems, Model Checking and Artificial Intelligence, Neural-Symbolic Learning and Reasoning, Plan, Activity, and Intent Recognition, Statistical Relational AI, Visual Representations and Reasoning, and Abstraction, Reformulation, and Approximation. This article presents short summaries of those events.


Reports of the AAAI 2010 Conference Workshops

AI Magazine

The AAAI-10 Workshop program was held Sunday and Monday, July 11–12, 2010 at the Westin Peachtree Plaza in Atlanta, Georgia. The AAAI-10 workshop program included 13 workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Fun, Bridging the Gap between Task and Motion Planning, Collaboratively-Built Knowledge Sources and Artificial Intelligence, Goal-Directed Autonomy, Intelligent Security, Interactive Decision Theory and Game Theory, Metacognition for Robust Social Systems, Model Checking and Artificial Intelligence, Neural-Symbolic Learning and Reasoning, Plan, Activity, and Intent Recognition, Statistical Relational AI, Visual Representations and Reasoning, and Abstraction, Reformulation, and Approximation. This article presents short summaries of those events.


An Ensemble Learning and Problem Solving Architecture for Airspace Management

AAAI Conferences

In this paper we describe the application of a novel learning and problem solving architecture to the domain of airspace management, where multiple requests for the use of airspace need to be reconciled and managed automatically. The key feature of our "Generalized Integrated Learning Architecture" (GILA) is a set of integrated learning and reasoning (ILR) systems coordinated by a central meta-reasoning executive (MRE). Each ILR learns independently from the same training example and contributes to problem-solving in concert with other ILRs as directed by the MRE. Formal evaluations show that our system performs as well as or better than humans after learning from the same training data. Further, GILA outperforms any individual ILR run in isolation, thus demonstrating the power of the ensemble architecture for learning and problem solving.


Goal-Driven Learning in the GILA Integrated Intelligence Architecture

AAAI Conferences

Goal Driven Learning (GDL) focuses on systems that determine by themselves what has to be learnt and how to learn it. Typically GDL systems use meta-reasoning capabilities over a base {\em reasoner}, identifying learning goals and devising strategies. In this paper we present a novel GDL technique to deal with complex AI systems where the meta-reasoning module has to analyze the reasoning trace of multiple components with potentially different learning paradigms. Our approach works by distributing the generation of learning strategies among the different modules instead of centralizing it in the meta-reasoner. We implemented our technique in the GILA system, that works in the airspace task orders domain, showing an increase in performance.


AAAI 1994 Spring Symposium Series Reports

AI Magazine

The Association for the Advancement of Artificial Intelligence (AAAI) held its 1994 Spring Symposium Series on 19-23 March at Stanford University, Stanford, California. This article contains summaries of 10 of the 11 symposia that were conducted: Applications of Computer Vision in Medical Image Processing; AI in Medicine: Interpreting Clinical Data; Believable Agents; Computational Organization Design; Decision-Theoretic Planning; Detecting and Resolving Errors in Manufacturing Systems; Goal-Driven Learning; Intelligent Multimedia, Multimodal Systems; Software Agents; and Toward Physical Interaction and Manipulation. Papers of most of the symposia are available as technical reports from AAAI.