This paper motivates and develops source traces for temporal difference (TD) learning in the tabular setting. Source traces are like eligibility traces, but model potential histories rather than immediate ones. This allows TD errors to be propagated to potential causal states and leads to faster generalization. Source traces can be thought of as the model-based, backward view of successor representations (SR), and share many of the same benefits. This view, however, suggests several new ideas. First, a TD(λ)-like source learning algorithm is proposed and its convergence is proven. Then, a novel algorithm for learning the source map (or SR matrix) is developed and shown to outperform the previous algorithm. Finally, various approaches to using the source/SR model are explored, and it is shown that source traces can be effectively combined with other model-based methods like Dyna and experience replay.

A touted use of Procedural Content Generation is generating content tailored to specific players. Previous work has relied on human identification of player profile features which are then mapped to level generator features. We present a machine-learned technique to train generators on Super Mario Bros. videos, generating levels based on latent play styles learned from the video. We evaluate the generators in comparison to the original levels and a machine-learned generator trained using simulated players.

The way people communicate — be it verbally, visually, or via text– is indicative of personality traits. In social media the concept of the status update is used for individuals to communicate to their social networks in an always-on fashion. In doing so individuals utilize various kinds of speech acts that, while primarily communicating their content, also leave traces of their personality dimensions behind. We human-coded a set of Facebook status updates from the myPersonality dataset in terms of speech acts label and then experimented with surface level linguistic features including lexical, syntactic, and simple sentiment detection to automatically label status updates as their appropriate speech act. We apply supervised learning to the dataset and using our features are able to classify with high accuracy two dominant kinds of acts that have been found to occur in social media. At the same time we used the coded data to perform a regression analysis to determine which speech acts are significant of certain personality dimensions. The implications of our work allow for automatic large-scale personality identification through social media status updates.

Emergency evacuations are dangerous situations for both evacuees and first responders. The use of automation in the form of guidance robots can reduce the danger to humans by both aiding evacuees and assisting first responders. This presents an interesting opportunity to explore the trust dynamic between frightened evacuees and automated robot guides. We present our work so far on designing robots to immediately generate trust as well as our initial concept of an algorithm for maintaining trust through interaction.

We propose a computational model of joint attention consisting of three parts: responding to joint attention, initiating joint attention, and ensuring joint attention. This model is supported by psychological findings and matches the developmental timeline in humans. We present two experiments that test this model and investigate joint attention in human-robot interaction. The first experiment explored the effects of responding to joint attention on human-robot interaction. We show that robots responding to joint attention are more transparent to humans and are more competent and socially interactive. The second experiment studied the importance of ensuring joint attention in human-robot interaction. Data upheld our hypotheses that a robot's ensuring joint attention behavior yields better performance in human-robot interactive tasks and ensuring joint attention behaviors are perceived as natural behaviors.