Discourse & Dialogue
eins.ai Next Generation Learning Suite For Completely Smart Classroom
Intelligence Assistants in classroom is the natural next step from interactive whiteboards and multiple software solutions. Offering classroom collaboration, next-level interactivity, curriculum content, grading, automatic attendance, sentiment analysis, and device management, eins.ai is truly comprehensive.
Learning to Memorize in Neural Task-Oriented Dialogue Systems
In this thesis, we leverage the neural copy mechanism and memory-augmented neural networks (MANNs) to address existing challenge of neural task-oriented dialogue learning. We show the effectiveness of our strategy by achieving good performance in multi-domain dialogue state tracking, retrieval-based dialogue systems, and generation-based dialogue systems. We first propose a transferable dialogue state generator (TRADE) that leverages its copy mechanism to get rid of dialogue ontology and share knowledge between domains. We also evaluate unseen domain dialogue state tracking and show that TRADE enables zero-shot dialogue state tracking and can adapt to new few-shot domains without forgetting the previous domains. Second, we utilize MANNs to improve retrieval-based dialogue learning. They are able to capture dialogue sequential dependencies and memorize long-term information. We also propose a recorded delexicalization copy strategy to replace real entity values with ordered entity types. Our models are shown to surpass other retrieval baselines, especially when the conversation has a large number of turns. Lastly, we tackle generation-based dialogue learning with two proposed models, the memory-to-sequence (Mem2Seq) and global-to-local memory pointer network (GLMP). Mem2Seq is the first model to combine multi-hop memory attention with the idea of the copy mechanism. GLMP further introduces the concept of response sketching and double pointers copying. We show that GLMP achieves the state-of-the-art performance on human evaluation.
Punchh Launches Deep Learning and Artificial Intelligence "Customer Sentiment Analysis" to Enable Real-Time Response to Customer Reviews
Punchh, the leader in digital marketing solutions for physical retailers, today announced the launch of Punchh Deep Sentiment Analysis. The new product allows brands to extract valuable insights from customer reviews using Punchh's natural language comprehension engine built with industry-leading deep learning and artificial intelligence. Its natural language processing model achieves human-level performance, defined as more than 93 percent accurate, and features multi-language support. "In today's hyper-competitive climate, brands need to do everything they can to foster and nurture direct customer relationships, and paying attention to customer reviews is an essential part of that," said Shyam Rao, CEO of Punchh. "Manually reading every review is prohibitively time-consuming for most retailers, which leads to slower response times and poor customer experiences. Our solution uses AI and machine learning to help brands analyze reviews at scale and immediately identify critical information so they can focus on high-level insights and make quick decisions to strengthen customer relationships and increase loyalty."
Automatic Evaluation of Local Topic Quality
Lund, Jeffrey, Armstrong, Piper, Fearn, Wilson, Cowley, Stephen, Byun, Courtni, Boyd-Graber, Jordan, Seppi, Kevin
Topic models are typically evaluated with respect to the global topic distributions that they generate, using metrics such as coherence, but without regard to local (token-level) topic assignments. Token-level assignments are important for downstream tasks such as classification. Even recent models, which aim to improve the quality of these token-level topic assignments, have been evaluated only with respect to global metrics. We propose a task designed to elicit human judgments of token-level topic assignments. We use a variety of topic model types and parameters and discover that global metrics agree poorly with human assignments. Since human evaluation is expensive we propose a variety of automated metrics to evaluate topic models at a local level. Finally, we correlate our proposed metrics with human judgments from the task on several datasets. We show that an evaluation based on the percent of topic switches correlates most strongly with human judgment of local topic quality. We suggest that this new metric, which we call consistency, be adopted alongside global metrics such as topic coherence when evaluating new topic models.
Identifying the Focus of Negation Using Discourse Structure
Sarabi, Zahra (University of North Texas) | Blanco, Eduardo (University of North Texas)
This paper presents experimental results showing that discourse structure is a useful element in identifying the focus of negation. We define features extracted from RST-like discourse trees. We experiment with the largest publicly available corpus and an off-the-shelf discourse parser. Results show that discourse structure is especially beneficial when predicting the focus of negations in long sentences.
Challenges in Building Intelligent Open-domain Dialog Systems
Huang, Minlie, Zhu, Xiaoyan, Gao, Jianfeng
There is a resurgent interest in developing intelligent open-domain dialog systems due to the availability of large amounts of conversational data and the recent progress on neural approaches to conversational AI. Unlike traditional task-oriented bots, an open-domain dialog system aims to establish long-term connections with users by satisfying the human need for communication, affection, and social belonging. This paper reviews the recent works on neural approaches that are devoted to addressing three challenges in developing such systems: semantics, consistency, and interactiveness. Semantics requires a dialog system to not only understand the content of the dialog but also identify user's social needs during the conversation. Consistency requires the system to demonstrate a consistent personality to win users trust and gain their long-term confidence. Interactiveness refers to the system's ability to generate interpersonal responses to achieve particular social goals such as entertainment, conforming, and task completion. The works we select to present here is based on our unique views and are by no means complete. Nevertheless, we hope that the discussion will inspire new research in developing more intelligent dialog systems.
Survey on Evaluation Methods for Dialogue Systems
Deriu, Jan, Rodrigo, Alvaro, Otegi, Arantxa, Echegoyen, Guillermo, Rosset, Sophie, Agirre, Eneko, Cieliebak, Mark
In this paper we survey the methods and concepts developed for the evaluation of dialogue systems. Evaluation is a crucial part during the development process. Often, dialogue systems are evaluated by means of human evaluations and questionnaires. However, this tends to be very cost and time intensive. Thus, much work has been put into finding methods, which allow to reduce the involvement of human labour. In this survey, we present the main concepts and methods. For this, we differentiate between the various classes of dialogue systems (task-oriented dialogue systems, conversational dialogue systems, and question-answering dialogue systems). We cover each class by introducing the main technologies developed for the dialogue systems and then by presenting the evaluation methods regarding this class.
A Modern Hands-On Approach to Sentiment Analysis - Synerzip
Human emotions are complex and difficult to decode. However, recent advancements in artificial intelligence and deep learning, are enabling new leaps in sentiment analysis. Put simply, sentiment analysis is a machine decoding human emotions for a specific purpose. Applications vary from mining opinions to gauging political inclinations to see how product reviews are affecting real-time sales. Social media companies actively use sentiment analysis to root out offensive and prejudiced content.
Where does active travel fit within local community narratives of mobility space and place?
Biehl, Alec, Chen, Ying, Sanabria-Veaz, Karla, Uttal, David, Stathopoulos, Amanda
Encouraging sustainable mobility patterns is at the forefront of policymaking at all scales of governance as the collective consciousness surrounding climate change continues to expand. Not every community, however, possesses the necessary economic or socio-cultural capital to encourage modal shifts away from private motorized vehicles towards active modes. The current literature on `soft' policy emphasizes the importance of tailoring behavior change campaigns to individual or geographic context. Yet, there is a lack of insight and appropriate tools to promote active mobility and overcome transport disadvantage from the local community perspective. The current study investigates the promotion of walking and cycling adoption using a series of focus groups with local residents in two geographic communities, namely Chicago's (1) Humboldt Park neighborhood and (2) suburb of Evanston. The research approach combines traditional qualitative discourse analysis with quantitative text-mining tools, namely topic modeling and sentiment analysis. The analysis uncovers the local mobility culture, embedded norms and values associated with acceptance of active travel modes in different communities. We observe that underserved populations within diverse communities view active mobility simultaneously as a necessity and as a symbol of privilege that is sometimes at odds with the local culture. The mixed methods approach to analyzing community member discourses is translated into policy findings that are either tailored to local context or broadly applicable to curbing automobile dominance. Overall, residents of both Humboldt Park and Evanston envision a society in which multimodalism replaces car-centrism, but differences in the local physical and social environments would and should influence the manner in which overarching policy objectives are met.
Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators
Yi, Sanghyun, Goel, Rahul, Khatri, Chandra, Chung, Tagyoung, Hedayatnia, Behnam, Venkatesh, Anu, Gabriel, Raefer, Hakkani-Tur, Dilek
Encoder-decoder based neural architectures serve as the basis of state-of-the-art approaches in end-to-end open domain dialog systems. Since most of such systems are trained with a maximum likelihood(MLE) objective they suffer from issues such as lack of generalizability and the generic response problem, i.e., a system response that can be an answer to a large number of user utterances, e.g., "Maybe, I don't know." Having explicit feedback on the relevance and interestingness of a system response at each turn can be a useful signal for mitigating such issues and improving system quality by selecting responses from different approaches. Towards this goal, we present a system that evaluates chatbot responses at each dialog turn for coherence and engagement. Our system provides explicit turn-level dialog quality feedback, which we show to be highly correlated with human evaluation. To show that incorporating this feedback in the neural response generation models improves dialog quality, we present two different and complementary mechanisms to incorporate explicit feedback into a neural response generation model: reranking and direct modification of the loss function during training. Our studies show that a response generation model that incorporates these combined feedback mechanisms produce more engaging and coherent responses in an open-domain spoken dialog setting, significantly improving the response quality using both automatic and human evaluation.