Discourse & Dialogue
How AI-Powered Sentiment Analysis Supercharges Your CX Strategy
While it's not uncommon for small and medium-sized businesses (SMBs) to switch financial institutions, the 2019 FIS Performance Against Customer Expectations report has found that the rate of churn is increasing. Historically, 13%-15% of small and medium-sized firms have been found to be actively reviewing their banking relationships. However, the turnover rate has now risen to 61% among the top 50 U.S. banks and 60% among regional banks. All it may take to push an already skeptical firm to switch is one more bad experience. So customer sentiment analysis could be exactly what financial institutions need to improve customer experience -- ideally, before things ever reach that pass.
Embedding Projection for Targeted Cross-lingual Sentiment: Model Comparisons and a Real-World Study
Barnes, Jeremy (University of Oslo) | Klinger, Roman
Sentiment analysis benefits from large, hand-annotated resources in order to train and test machine learning models, which are often data hungry. While some languages, e.g., English, have a vast arrayof these resources, most under-resourced languages do not, especially for fine-grained sentiment tasks, such as aspect-level or targeted sentiment analysis. To improve this situation, we propose a cross-lingual approach to sentiment analysis that is applicable to under-resourced languages and takes into account target-level information. This model incorporates sentiment information into bilingual distributional representations, byjointly optimizing them for semantics and sentiment, showing state-of-the-art performance at sentence-level when combined with machine translation. The adaptation to targeted sentiment analysis on multiple domains shows that our model outperforms other projection-based bilingual embedding methods on binary targetedsentiment tasks. Our analysis on ten languages demonstrates that the amount of unlabeled monolingual data has surprisingly little effect on the sentiment results. As expected, the choice of a annotated source language for projection to a target leads to better results for source-target language pairs which are similar. Therefore, our results suggest that more efforts should be spent on the creation of resources for less similar languages tothose which are resource-rich already. Finally, a domain mismatch leads to a decreased performance. This suggests resources in any language should ideally cover varieties of domains.
Prediction Focused Topic Models for Electronic Health Records
Ren, Jason, Kunes, Russell, Doshi-Velez, Finale
Electronic Health Record (EHR) data can be represented as di screte counts over a high dimensional set of possible procedures, diagnoses, a nd medications. Supervised topic models present an attractive option for inco rporating EHR data as features into a prediction problem: given a patient's recor d, we estimate a set of latent factors that are predictive of the response variab le. However, existing methods for supervised topic modeling struggle to balance p rediction quality and coherence of the latent factors. W e introduce a novel approa ch, the prediction-focused topic model, that uses the supervisory signal to ret ain only features that improve, or do not hinder, prediction performance. By remov ing features with irrelevant signal, the topic model is able to learn task-relev ant, interpretable topics. W e demonstrate on a EHR dataset and a movie review dataset tha t compared to existing approaches, prediction-focused topic models are able to learn much more coherent topics while maintaining competitive prediction s.
Improving Robustness of Task Oriented Dialog Systems
Einolghozati, Arash, Gupta, Sonal, Mohit, Mrinal, Shah, Rushin
Task oriented language understanding in dialog systems is often modeled using intents (task of a query) and slots (parameters for that task). Intent detection and slot tagging are, in turn, modeled using sentence classification and word tagging techniques respectively. Similar to adversarial attack problems with computer vision models discussed in existing literature, these intent-slot tagging models are often over-sensitive to small variations in input -- predicting different and often incorrect labels when small changes are made to a query, thus reducing their accuracy and reliability. However, evaluating a model's robustness to these changes is harder for language since words are discrete and an automated change (e.g. adding `noise') to a query sometimes changes the meaning and thus labels of a query. In this paper, we first describe how to create an adversarial test set to measure the robustness of these models. Furthermore, we introduce and adapt adversarial training methods as well as data augmentation using back-translation to mitigate these issues. Our experiments show that both techniques improve the robustness of the system substantially and can be combined to yield the best results.
167: Artificial Intelligence and Customer Sentiment from Everyday MBA
Episode 167 - Kevin Craine and Billee Howard discuss the use of nuero-powered technology to quantify, measure and understand human thought. Explore how to use artificial intelligence and sentiment analysis to connect customer emotion directly to improved business performance. Understand the convergence of'big emotion' and'big data' and how it is valuable from a strategic and marketing perspective. Stay tuned for three action items in the second half. Do you want to be a guest?
xSLUE: A Benchmark and Analysis Platform for Cross-Style Language Understanding and Evaluation
Every natural text is written in some style. The style is formed by a complex combination of different stylistic factors, including formality markers, emotions, metaphors, etc. Some factors implicitly reflect the author's personality, while others are explicitly controlled by the author's choices in order to achieve some personal or social goal. One cannot form a complete understanding of a text and its author without considering these factors. The factors combine and co-vary in complex ways to form styles. Studying the nature of the covarying combinations sheds light on stylistic language in general, sometimes called cross-style language understanding. This paper provides a benchmark corpus (xSLUE) with an online platform (http://xslue.com) for cross-style language understanding and evaluation. The benchmark contains text in 15 different styles and 23 classification tasks. For each task, we provide the fine-tuned classifier for further analysis. Our analysis shows that some styles are highly dependent on each other (e.g., impoliteness and offense), and some domains (e.g., tweets, political debates) are stylistically more diverse than others (e.g., academic manuscripts). We discuss the technical challenges of cross-style understanding and potential directions for future research: cross-style modeling which shares the internal representation for low-resource or low-performance styles and other applications such as cross-style generation.
Investigation of Error Simulation Techniques for Learning Dialog Policies for Conversational Error Recovery
Fazel-Zarandi, Maryam, Wang, Longshaokan, Tiwari, Aditya, Matsoukas, Spyros
Training dialog policies for speech-based virtual assistants requires a plethora of conversational data. The data collection phase is often expensive and time consuming due to human involvement. To address this issue, a common solution is to build user simulators for data generation. For the successful deployment of the trained policies into real world domains, it is vital that the user simulator mimics realistic conditions. In particular, speech-based assistants are heavily affected by automatic speech recognition and language understanding errors, hence the user simulator should be able to simulate similar errors. In this paper, we review the existing error simulation methods that induce errors at audio, phoneme, text, or semantic level; and conduct detailed comparisons between the audio-level and text-level methods. In the process, we improve the existing text-level method by introducing confidence score prediction and out-of-vocabulary word mapping. We also explore the impact of audio-level and text-level methods on learning a simple clarification dialog policy to recover from errors to provide insight on future improvement for both approaches.
dotnet/machinelearning-samples
The goal is to be able to make SENTIMENT ANALYSIS prediction/detection of what the user is writing in a very UI interactive app (BLAZOR based) in the client side and running an ML.NET model (Sentiment analysis based on binary-classification) in the server side. From ML.NET perspective, the goal is to optimize the ML.NET model executions in the server by sharing the ML.NET objects used for predictions across Http requests and being able to implement very simple code to be used by the user when predicting, like the following line of code that you could write on any ASP.NET Core controller's method or custom service class: The object predictionEnginePool will be injected in the controller's constructor or into you custom class. Internally, it is optimized so the object dependencies are cached and shared across Http requests with minimum overhead when creating those objects. Blazor allows you to run C# code in the client side, as shown in the architecture figure. For this sample we chose to run the ML.NET model in the server side, so the model is protected within the service.
Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering
Multi-domain dialogue state tracking (DST) is a critical component for conversational AI systems. The domain ontology (i.e., specification of domains, slots, and values) of a conversational AI system is generally incomplete, making the capability for DST models to generalize to new slots, values, and domains during inference imperative. In this paper, we propose to model multi-domain DST as a question answering problem, referred to as Dialogue State Tracking via Question Answering (DSTQA). Within DSTQA, each turn generates a question asking for the value of a (domain, slot) pair, thus making it naturally extensible to unseen domains, slots, and values. Additionally, we use a dynamically-evolving knowledge graph to explicitly learn relationships between (domain, slot) pairs. Our model has a 5.80% and 12.21% relative improvement over the current state-of-the-art model on MultiWOZ 2.0 and MultiWOZ 2.1 datasets, respectively. Additionally, our model consistently outperforms the state-of-the-art model in domain adaptation settings.