A new version (v0.9.0) has recently been released, so we use this as an opportunity to play with some new functionality. The goal of today's post will be to perform sentiment analysis on movie reviews from IMDB. Note: ML.NET is still evolving, this post was written using Microsoft.ML v0.9.0. If you don't have it installed, head out to the .NET Core Downloads page. Tangential, but you can also get here by going to dot.net, then navigating to Downloads and .NET Core.

The ability to accurately track what happens during a conversation is essential for the performance of a dialogue system. Current state-of-the-art multi-domain dialogue state trackers achieve just over 55% accuracy on the current go-to benchmark, which means that in almost every second dialogue turn they place full confidence in an incorrect dialogue state. Belief trackers, on the other hand, maintain a distribution over possible dialogue states. However, they lack in performance compared to dialogue state trackers, and do not produce well calibrated distributions. In this work we present state-of-the-art performance in calibration for multi-domain dialogue belief trackers using a calibrated ensemble of models. Our resulting dialogue belief tracker also outperforms previous dialogue belief tracking models in terms of accuracy.

Key Phrase Extraction: Using this function, you can feed big chunks of unstructured text to the system and get a list of key phrases. Unlike sentiment analysis, this function can deliver better results if you provide text in bigger blocks. Language Detection: This function analyzes the input text and provides the ISO identifier and language name. It can be leveraged to evaluate data columns in which the language of the text in not known. Currently about 120 languages are supported. Image Tagging: This function supports tagging of 2000 recognizable objects, living species, environmental settings, and actions.

Sentiment Analysis can be defined as the process of analyzing text data and categorizing them into Positive, Negative, or Neutral sentiments. Sentiment Analysis is used in many cases like Social Media Monitoring, Customer service, Brand Monitoring, political campaigns, etc. Analyzing customer feedback such as social media conversations, product reviews, and survey responses allows companies to understand the customer's emotions better which is becoming more essential to meet their needs. It is almost impossible to manually sort thousands of social media conversations, customer reviews, and surveys. So we have to use either ML/DL to build a model that analyzes the text data and performs the required operations. The problem I am trying to solve here is part of this Kaggle competition.

Existing dialogue state tracking (DST) models require plenty of labeled data. However, collecting high-quality labels is costly, especially when the number of domains increases. In this paper, we address a practical DST problem that is rarely discussed, i.e., learning efficiently with limited labeled data. We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior. We encourage a DST model to have consistent latent distributions given a perturbed input, making it more robust to an unseen scenario. We also add an auxiliary utterance generation task, modeling a potential correlation between conversational behavior and dialogue states. The experimental results show that our proposed self-supervised signals can improve joint goal accuracy by 8.95\% when only 1\% labeled data is used on the MultiWOZ dataset. We can achieve an additional 1.76\% improvement if some unlabeled data is jointly trained as semi-supervised learning. We analyze and visualize how our proposed self-supervised signals help the DST task and hope to stimulate future data-efficient DST research.

We investigate the problem of multi-domain Dialogue State Tracking (DST) with open vocabulary. Existing approaches exploit BERT encoder and copy-based RNN decoder, where the encoder first predicts the state operation, and then the decoder generates new slot values. However, in this stacked encoder-decoder structure, the operation prediction objective only affects the BERT encoder and the value generation objective mainly affects the RNN decoder. In this paper, we propose a purely Transformer-based framework that uses BERT as both encoder and decoder. In so doing, the operation prediction objective and the value generation objective can jointly optimize our model for DST. At the decoding step, we re-use the hidden states of the encoder in the self-attention mechanism of the corresponding decoder layer to construct a flat model structure for effective parameter updating. Experimental results show that our approach substantially outperforms the existing state-of-the-art framework, and it also achieves very competitive performance to the best ontology-based approaches.

The migration of conversational AI agents across different embodiments in order to maintain the continuity of the task has been recently explored to further improve user experience. However, these migratable agents lack contextual understanding of the user information and the migrated device during the dialog conversations with the user. This opens the question of how an agent might behave when migrated into an embodiment for contextually predicting the next utterance. We collected a dataset from the dialog conversations between crowdsourced workers with the migration context involving personal and non-personal utterances in different settings (public or private) of embodiment into which the agent migrated. We trained the generative and information retrieval models on the dataset using with and without migration context and report the results of both qualitative metrics and human evaluation. We believe that the migration dataset would be useful for training future migratable AI systems.

We investigate the problem of multi-domain Dialogue State Tracking (DST) with open vocabulary, which aims to extract the state from the dialogue. Existing approaches usually concatenate previous dialogue state with dialogue history as the input to a bi-directional Transformer encoder. They rely on the self-attention mechanism of Transformer to connect tokens in them. However, attention may be paid to spurious connections, leading to wrong inference. In this paper, we propose to construct a dialogue state graph in which domains, slots and values from the previous dialogue state are connected properly. Through training, the graph node and edge embeddings can encode co-occurrence relations between domain-domain, slot-slot and domain-slot, reflecting the strong transition paths in general dialogue. The state graph, encoded with relational-GCN, is fused into the Transformer encoder. Experimental results show that our approach achieves a new state of the art on the task while remaining efficient. It outperforms existing open-vocabulary DST approaches.

A dialogue system for disease diagnosis aims at making a diagnosis by conversing with patients. Existing disease diagnosis dialogue systems highly rely on data-driven methods and statistical features, lacking profound comprehension of medical knowledge, such as symptom-disease relations. In addition, previous work pays less attention to demographic attributes of a patient, which are important factors in clinical diagnoses. To tackle these issues, this work presents a graph based and demographic attributes aware dialogue system for disease diagnosis. Specifically, we first build a weighted bidirectional graph based on clinical dialogues to depict the relationship between symptoms and diseases and then present a bidirectional graph based deep Q-network (BG-DQN) for dialogue management. By extending Graph Convolutional Network (GCN) to learn the embeddings of diseases and symptoms from both the structural and attribute information in the graph, BG-DQN could capture the relations between diseases and symptoms better. Moreover, BG-DQN also encodes the demographic attributes of a patient to assist the disease diagnosis process. Experimental results show that the proposed dialogue system outperforms several competitive methods in terms of diagnostic accuracy. More importantly, our method can complete the task with less dialogue turns and possesses better distinguishing capability on diseases with similar symptoms.

We propose a framework to integrate the concept of Theory of Mind (ToM) into generating utterances for task-oriented dialogue. Our approach explores the ability to model and infer personality types of opponents, predicts their responses, and uses this information to adapt the agent's high-level strategy in negotiation tasks. We introduce a probabilistic formulation for the first-order theory of mind and test our approach on the CraigslistBargain dataset. Experiments show that our method using ToM inference achieves a 40\% higher dialogue agreement rate compared to baselines on a mixed population of opponents. We also show that our model displays diverse negotiation behavior with different types of opponents.