Human communication is multimodal and includes elements such as gesture and facial expression along with spoken language. Modern technology makes it feasible to capture all such aspects of communication in natural settings. As a result, similar to fields such as genetics, astronomy and neuroscience, scholars in areas such as linguistics and communication studies are on the verge of a data-driven revolution in their fields. These new approaches require analytical support from machine learning and artificial intelligence to develop tools to help process the vast data repositories. The Distributed Little Red Hen Lab project is an international team of interdisciplinary researchers building a large-scale infrastructure for data-driven multimodal communications research. In this paper, we describe a machine learning system developed to automatically annotate a large database of television program videos as part of this project. The annotations mark regions where people or speakers are on screen along with body part motions including head, hand and shoulder motion. We also annotate a specific class of gestures known as timeline gestures. An existing gesture annotation tool, ELAN, can be used with these annotations to quickly locate gestures of interest. Finally, we provide an update mechanism for the system based on human feedback. We empirically evaluate the accuracy of the system as well as present data from pilot human studies to show its effectiveness at aiding gesture scholars in their work.

This work explores the possibility of learning word order before syntactic concepts such as subject, object, or lexical A hallmark of human word learning is the integration categories or syntactic parse representations are available of cross-situational information even though this information to the learner. It also examines the utility of the acquired is not always reliable as inconsistencies in the wordreferent word order in a joint learner where word order knowledge co-occurrence (e.g., when the referent is absent in a constrains word learning (syntactic bootstrapping) and vice scene or when distracting referents are present) inject noise versa. We propose that the transitional probabilities of the into cross-situational information. It has been suggested thematic roles (in the order of their appearance in the utterance) that bootstrapping cross-situational word learning with the of the referential words (words with action or event learner's belief about the referential intentions of the speaker participant referents) are an invaluable source of information (Frank, Goodman, and Tenenbaum 2009) as well as bootstrapping for learning word order and that they can provide an it with learner's belief about the syntactic regularities initial understanding of the notion of word order in early of language (Yu 2006; Maurits, Perfors, and Navarro stages of language acquisition in the absence of advanced 2009; Alishahi and Fazly 2010; Alishahi and Chrupała 2012; syntactic concepts or representations. We utilize an incremental Abend et al. 2017) allow for disambiguation and should and memory-limited learning algorithm as opposed thus improve word learning. Maurits, Perfors, and Navarro to batch learning algorithms, as we are interested in online (2009) bootstrap word learning with the acquired knowledge learning in embodied agents with computational limitations. of word order in an ideal learner although their model cannot Our model adds the notion of syntax to the word learning

In this paper we study a new learning problem defined as "Single-Teacher Multi-Student" (STMS) problem, which investigates how to learn a series of student (simple and specific) models from a single teacher (complex and universal) model. Taking the multiclass and binary classification for example, we focus on learning multiple binary classifiers from a single multiclass classifier, where each of binary classifier is responsible for a certain class. This actually derives from some realistic problems, such as identifying the suspect based on a comprehensive face recognition system. By treating the already-trained multiclass classifier as the teacher, and multiple binary classifiers as the students, we propose a gated support vector machine (gSVM) as a solution. A series of gSVMs are learned with the help of single teacher multiclass classifier. The teacher's help is two-fold; first, the teacher's score provides the gated values for students' decision; second, the teacher can guide the students to accommodate training examples with different difficulty degrees. Extensive experiments on real datasets validate its effectiveness.

We propose a learning-based system for detecting when a user performs a pointing gesture, using data acquired from IMU sensors, by means of a 1D convolutional neural network. We quantitatively evaluate the resulting detection accuracy, and discuss an application to a human-robot interaction task where pointing gestures are used to guide a quadrotor landing.

Twitter has become an important online source for real-time news dissemination. Especially, official accounts of local government and media outlets have provided newsworthy and authoritative information, revealing local trends and breaking news. In this paper, we describe TipMaster an automatically constructed knowledge base of Twitter accounts that are likely to report local news, from government agencies to local media outlets. First, we implement classifiers for detecting these accounts by integrating heterogeneous information from the accounts' textual metadata, profile images, and their tweet messages. Next, we demonstrate two use cases for TipMaster: 1) as a platform that monitors real-time social media messages for local breaking news, and 2) as an authoritative source for verifying nascent rumors. Experimental results show that our account classification algorithms achieve both high precision and recall (around 90%). The demonstrated case studies prove that our platform is able to detect local breaking news or debunk emergent rumors faster than mainstream media sources.

As the demand for mobile network services increases, immediate detection and forecasting of network failure events have become important problems for service providers. Several event detection approaches have been proposed to tackle these problems by utilizing social data. However, these approaches have not tried to solve event detection and forecasting problems from multiple data sets, such as web access logs and search queries. In this paper, we propose a machine learning approach that incorporates multiple user activity data into detecting and forecasting failure events. Our approach is based on a two-level procedure. First, we introduce a novel feature construction method that treats both the imbalanced label problem and the data sparsity problem of user activity data. Second, we propose a model ensemble method that combines outputs of supervised and unsupervised learning models for each data set and gives accurate predictions of network service outage. We demonstrate the effectiveness of the proposed models by extensive experiments with real-world failure events occurred at a network service provider in Japan and three user activity data sets.

In most cities, taxis play an important role in providing point-to-point transportation service. If the taxi service is reliable, responsive, and cost-effective, past studies show that taxi-like services can be a viable choice in replacing a significant amount of private cars. However, making taxi services efficient is extremely challenging, mainly due to the fact that taxi drivers are self-interested and they operate with only local information. Although past research has demonstrated how recommendation systems could potentially help taxi drivers in improving their performance, most of these efforts are not feasible in practice. This is mostly due to the lack of both the comprehensive data coverage and an efficient recommendation engine that can scale to tens of thousands of drivers. In this paper, we propose a comprehensive working platform called the Driver Guidance System (DGS). With real-time citywide taxi data provided by our collaborator in Singapore, we demonstrate how we can combine real-time data analytics and large-scale optimization to create a guidance system that can potentially benefit tens of thousands of taxi drivers. Via a realistic agent-based simulation, we demonstrate that drivers following DGS can significantly improve their performance over ordinary drivers, regardless of the adoption ratios. We have concluded our system designing and building and have recently entered the field trial phase.

Customer Relationship Management (CRM) systems play an important role in helping companies identify and keep sales and service prospects. CRM service providers offer a range of tools and techniques that will help find, sell to and keep customers. To be effective, CRM users usually require extensive training. Predictive CRM using machine learning expands the capabilities of traditional CRM through the provision of predictive insights for CRM users by combining internal and external data. In this paper, we will explore a novel idea of computationally learning salesmanship, its patterns and success factors to drive industry intuitions for a more predictable road to a vehicle sale. The newly discovered patterns and insights are used to act as a virtual guide or trainer for the general CRM user population.

Death is an inevitable part of life and while it cannot be delayed indefinitely it is possible to predict with some certainty when the health of a person is going to deteriorate. In this paper, we predict risk of mortality for patients from two large hospital systems in the Pacific Northwest. Using medical claims and electronic medical records (EMR) data we greatly improve prediction for risk of mortality and explore machine learning models with explanations for end of life predictions. The insights that are derived from the predictions can then be used to improve the quality of patient care towards the end of life.

Massive classification, a classification task defined over a vast number of classes (hundreds of thousands or even millions), has become an essential part of many real-world systems, such as face recognition. Existing methods, including the deep networks that achieved remarkable success in recent years, were mostly devised for problems with a moderate number of classes. They would meet with substantial difficulties, e.g., excessive memory demand and computational cost, when applied to massive problems. We present a new method to tackle this problem. This method can efficiently and accurately identify a small number of "active classes" for each mini-batch, based on a set of dynamic class hierarchies constructed on the fly. We also develop an adaptive allocation scheme thereon, which leads to a better tradeoff between performance and cost. On several large-scale benchmarks, our method significantly reduces the training cost and memory demand, while maintaining competitive performance.