Discourse & Dialogue
The Adapter-Bot: All-In-One Controllable Conversational Model
Madotto, Andrea, Lin, Zhaojiang, Bang, Yejin, Fung, Pascale
Considerable progress has been made towards conversational models that generate coherent and fluent responses by training large language models on large dialogue datasets. These models have little or no control of the generated responses and miss two important features: continuous dialogue skills integration and seamlessly leveraging diverse knowledge sources. In this paper, we propose the Adapter-Bot, a dialogue model that uses a fixed backbone conversational model such as DialGPT (Zhang et al., 2019) and triggers on-demand dialogue skills (e.g., emphatic response, weather information, movie recommendation) via different adapters (Houlsby et al., 2019). Each adapter can be trained independently, thus allowing a continual integration of skills without retraining the entire model. Depending on the skills, the model is able to process multiple knowledge types, such as text, tables, and graphs, in a seamless manner. The dialogue skills can be triggered automatically via a dialogue manager, or manually, thus allowing high-level control of the generated responses. At the current stage, we have implemented 12 response styles (e.g., positive, negative etc.), 8 goal-oriented skills (e.g. weather information, movie recommendation, etc.), and personalized and emphatic responses. We evaluate our model using automatic evaluation by comparing it with existing state-of-the-art conversational models, and we have released an interactive system at adapter.bot.ust.hk.
Cross-language sentiment analysis of European Twitter messages duringthe COVID-19 pandemic
Kruspe, Anna, Häberle, Matthias, Kuhn, Iona, Zhu, Xiao Xiang
Social media data can be a very salient source of information during crises. User-generated messages provide a window into people's minds during such times, allowing us insights about their moods and opinions. Due to the vast amounts of such messages, a large-scale analysis of population-wide developments becomes possible. In this paper, we analyze Twitter messages (tweets) collected during the first months of the COVID-19 pandemic in Europe with regard to their sentiment. This is implemented with a neural network for sentiment analysis using multilingual sentence embeddings. We separate the results by country of origin, and correlate their temporal development with events in those countries. This allows us to study the effect of the situation on people's moods. We see, for example, that lockdown announcements correlate with a deterioration of mood in almost all surveyed countries, which recovers within a short time span.
A Fast and Robust BERT-based Dialogue State Tracker for Schema-Guided Dialogue Dataset
Noroozi, Vahid, Zhang, Yang, Bakhturina, Evelina, Kornuta, Tomasz
Dialog State Tracking (DST) is one of the most crucial modules for goal-oriented dialogue systems. In this paper, we introduce FastSGT (Fast Schema Guided Tracker), a fast and robust BERT-based model for state tracking in goal-oriented dialogue systems. The proposed model is designed for the Schema-Guided Dialogue (SGD) dataset which contains natural language descriptions for all the entities including user intents, services, and slots. The model incorporates two carry-over procedures for handling the extraction of the values not explicitly mentioned in the current user utterance. It also uses multi-head attention projections in some of the decoders to have a better modelling of the encoder outputs. In the conducted experiments we compared FastSGT to the baseline model for the SGD dataset. Our model keeps the efficiency in terms of computational and memory consumption while improving the accuracy significantly. Additionally, we present ablation studies measuring the impact of different parts of the model on its performance. We also show the effectiveness of data augmentation for improving the accuracy without increasing the amount of computational resources.
Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalance
Yilmaz, Selim F., Kaynak, E. Batuhan, Koç, Aykut, Dibeklioğlu, Hamdi, Kozat, Suleyman S.
We investigate cross-lingual sentiment analysis, which has attracted significant attention due to its applications in various areas including market research, politics and social sciences. In particular, we introduce a sentiment analysis framework in multi-label setting as it obeys Plutchik wheel of emotions. We introduce a novel dynamic weighting method that balances the contribution from each class during training, unlike previous static weighting methods that assign non-changing weights based on their class frequency. Moreover, we adapt the focal loss that favors harder instances from single-label object recognition literature to our multi-label setting. Furthermore, we derive a method to choose optimal class-specific thresholds that maximize the macro-f1 score in linear time complexity. Through an extensive set of experiments, we show that our method obtains the state-of-the-art performance in 7 of 9 metrics in 3 different languages using a single model compared to the common baselines and the best-performing methods in the SemEval competition. We publicly share our code for our model, which can perform sentiment analysis in 100 languages, to facilitate further research.
CareCall: a Call-Based Active Monitoring Dialog Agent for Managing COVID-19 Pandemic
Lee, Sang-Woo, Jung, Hyunhoon, Ko, SukHyun, Kim, Sunyoung, Kim, Hyewon, Doh, Kyoungtae, Park, Hyunjung, Yeo, Joseph, Ok, Sang-Houn, Lee, Joonhaeng, Lim, Sungsoon, Jeong, Minyoung, Choi, Seongjae, Hwang, SeungTae, Park, Eun-Young, Ma, Gwang-Ja, Han, Seok-Joo, Cha, Kwang-Seung, Sung, Nako, Ha, Jung-Woo
Tracking suspected cases of COVID-19 is crucial to suppressing the spread of COVID-19 pandemic. Active monitoring and proactive inspection are indispensable to mitigate COVID-19 spread, though these require considerable social and economic expense. To address this issue, we introduce CareCall, a call-based dialog agent which is deployed for active monitoring in Korea and Japan. We describe our system with a case study with statistics to show how the system works. Finally, we discuss a simple idea which uses CareCall to support proactive inspection.
Learning from students' perception on professors through opinion mining
Vargas-Calderón, Vladimir, Flórez, Juan S., Ardila, Leonel F., Parra-A., Nicolas, Camargo, Jorge E., Vargas, Nelson
Students' perception of classes measured through their opinions on teaching surveys allows to identify deficiencies and problems, both in the environment and in the learning methodologies. The purpose of this paper is to study, through sentiment analysis using natural language processing (NLP) and machine learning (ML) techniques, those opinions in order to identify topics that are relevant for students, as well as predicting the associated sentiment via polarity analysis. As a result, it is implemented, trained and tested two algorithms to predict the associated sentiment as well as the relevant topics of such opinions. The combination of both approaches then becomes useful to identify specific properties of the students' opinions associated with each sentiment label (positive, negative or neutral opinions) and topic. Furthermore, we explore the possibility that students' perception surveys are carried out without closed questions, relying on the information that students can provide through open questions where they express their opinions about their classes.
Portfolio
The Linguistic Universe of Hungarian Poet Endre Ady Gender Stereotypes of Hungarian Online Media Named Entities in Hungarian Online Media Growth Hacking with NLP and Sentiment Analysis - our 5-week course at Manning Publications Metaphor and National Identity Alternative conceptualization of the Treaty of Trianon - 2019, John Benjamins Publishing Company We helped the future…
SentiQ: A Probabilistic Logic Approach to Enhance Sentiment Analysis Tool Quality
Kouadri, Wissam Maamar, Benbernou, Salima, Ouziri, Mourad, Palpanas, Themis, Amor, Iheb Ben
The opinion expressed in various Web sites and social-media is an essential contributor to the decision making process of several organizations. Existing sentiment analysis tools aim to extract the polarity (i.e., positive, negative, neutral) from these opinionated contents. Despite the advance of the research in the field, sentiment analysis tools give \textit{inconsistent} polarities, which is harmful to business decisions. In this paper, we propose SentiQ, an unsupervised Markov logic Network-based approach that injects the semantic dimension in the tools through rules. It allows to detect and solve inconsistencies and then improves the overall accuracy of the tools. Preliminary experimental results demonstrate the usefulness of SentiQ.
A Visual Introduction to Machine Learning - Machine Learning
Using NLP and sentiment analysis dictionaries, different features are computed. NLP and sentiment analysis is a must for the visual introduction of machine learning. A brief feature engineering is performed to get realistic results. Out of all computed features, the most outperformed features are selected for the Machine Learning model. The outperformed features are computed using various techniques that include information gain, gain ratio, and correlation score.
Are Neural Open-Domain Dialog Systems Robust to Speech Recognition Errors in the Dialog History? An Empirical Study
Gopalakrishnan, Karthik, Hedayatnia, Behnam, Wang, Longshaokan, Liu, Yang, Hakkani-Tur, Dilek
Large end-to-end neural open-domain chatbots are becoming increasingly popular. However, research on building such chatbots has typically assumed that the user input is written in nature and it is not clear whether these chatbots would seamlessly integrate with automatic speech recognition (ASR) models to serve the speech modality. We aim to bring attention to this important question by empirically studying the effects of various types of synthetic and actual ASR hypotheses in the dialog history on TransferTransfo, a state-of-the-art Generative Pre-trained Transformer (GPT) based neural open-domain dialog system from the NeurIPS ConvAI2 challenge. We observe that TransferTransfo trained on written data is very sensitive to such hypotheses introduced to the dialog history during inference time. As a baseline mitigation strategy, we introduce synthetic ASR hypotheses to the dialog history during training and observe marginal improvements, demonstrating the need for further research into techniques to make end-to-end open-domain chatbots fully speech-robust. To the best of our knowledge, this is the first study to evaluate the effects of synthetic and actual ASR hypotheses on a state-of-the-art neural open-domain dialog system and we hope it promotes speech-robustness as an evaluation criterion in open-domain dialog.