Anomaly detection methods abound and are used extensively in streaming settings in a wide variety of domains. But a strength can also be a weakness; given the vast number of methods, how can one select the best method for their application? Unfortunately, there is no one best way for all domains. Existing literature is focused on creating new anomaly detection methods or creating large frameworks for experimenting with multiple methods at the same time. As the literature continues to grow, extensive evaluation of every available anomaly detection method is not feasible. To reduce this evaluation burden, in this paper we present a framework to intelligently choose the optimal anomaly detection methods based on the characteristics the time series displays. We provide a comprehensive experimental validation of multiple anomaly detection methods over different time series characteristics to form guidelines. Applying our framework can save time and effort by surfacing the most promising anomaly detection methods instead of experimenting extensively with a rapidly expanding library of anomaly detection methods.

In (Yang et al. 2016), a hierarchical attention network (HAN) is created for document classification. The attention layer can be used to visualize text influential in classifying the document, thereby explaining the model's prediction. We successfully applied HAN to a sequential analysis task in the form of real-time monitoring of turn taking in conversations. However, we discovered instances where the attention weights were uniform at the stopping point (indicating all turns were equivalently influential to the classifier), preventing meaningful visualization for real-time human review or classifier improvement. We observed that attention weights for turns fluctuated as the conversations progressed, indicating turns had varying influence based on conversation state. Leveraging this observation, we develop a method to create more informative real-time visuals (as confirmed by human reviewers) in cases of uniform attention weights using the changes in turn importance as a conversation progresses over time.

Existing research on escalation recommendation often relies on acoustic features, obtainable in Spoken Dialog Systems (SDS). It is less understood how multi-modal dialog systems can recommend escalations online without access to such features. Several machine learning techniques are evaluated using only text features common to all dialog systems. We then present and implement a general criteria for online escalation recommendation based on the conversation structure, presence of correctional language, user request repetition, user intent, and polarity. Our method is designed to work with any automated assistant using text or speech input, even where inputs can alternate between text and speech. We achieve higher precision using less training data than several standard machine learning techniques on a dataset consisting of 7,754 conversations with live multi-modal automated assistants.

When reviewing a chatbot's performance, it is desirable to prioritize conversations involving misunderstood human inputs. A system for measuring the posthoc risk of missed intent associated with a single human input is presented. Using defined indicators of risk, the system's performance in identifying misunderstood human inputs is given. These indicators are given weights and optimized on real world data. By application of our system, language model development is improved.

Causal structure learning from time series data is a major scientific challenge. Existing algorithms assume that measurements occur sufficiently quickly; more precisely, they assume that the system and measurement timescales are approximately equal. In many scientific domains, however, measurements occur at a significantly slower rate than the underlying system changes. Moreover, the size of the mismatch between timescales is often unknown. This paper provides three distinct causal structure learning algorithms, all of which discover all dynamic graphs that could explain the observed measurement data as arising from undersampling at some rate. That is, these algorithms all learn causal structure without assuming any particular relation between the measurement and system timescales; they are thus rate-agnostic. We apply these algorithms to data from simulations. The results provide insight into the challenge of undersampling.