Adversarial behavioral has been observed in many different contexts. In this paper we address the problem of adversarial behavior in the context of clandestine networks. We use data from a massively multiplayer online role playing game to illustrate the behavioral and structural signatures of deviant players change over time as a response to "policing" activities of the game administrators. Preliminary results show that the behavior of the deviant players and their affiliates show co-evolutionary behavior and the timespan within the game can be divided into different epochs based on their behaviors. Feature sets derived from these results can be used for better predictive machine learning models for detecting deviants in clandestine networks.

Automatic detection of persuasion is essential for machine interaction on the social web. To facilitate automated persuasion detection, we present a novel microtext corpus derived from hostage negotiation transcripts as well as a detailed manual (codebook) for persuasion annotation. Our corpus, called the NPS Persuasion Corpus, consists of 37 transcripts from four sets of hostage negotiation transcriptions. Each utterance in the corpus is hand annotated for one of nine categories of persuasion based on Cialdini’s model: reciprocity, commitment, consistency, liking, authority, social proof, scarcity, other, and not persuasive. Initial results using three supervised learning algorithms (Na ̈ve Bayes, Maximum Entropy, and Support Vector Machines) combined with gappy and orthogonal sparse bigram feature expansion techniques show that the annotation process did capture machine learnable features of persuasion with F-scores better than baseline.

In this paper, we study a time-critical facet of context-awareness: context transitions, which we model as changes in specific context types over time, e.g., activity or location. We present results from a user-centred field study involving participant interviews and motion data capture from two mobile device sensors: the accelerometer and magnetic field sensor. The results show how the participants subjectively interpret their daily context transitions with variable granularity, and a comparison of these context transitions with mobile device motion data shows how the motion data poorly reflect the identified transitions. The results imply that care should be taken when representing and modelling users’ subjective interpretations of context, as well as the objective nature of context sensors. Furthermore, processing and usability trade-offs should be made if real-time on-device transition detection is to be implemented.

We analyze the patterns of autistic reasoning while performing planning tasks. The formalism of non-monotonic logic of defaults is used to simulate the autistic decision-making while adjusting an action to a context. Our current main finding is that while people with autism may be able to process single default rules, they have a characteristic difficulty in cases where multiple default rules conflict. Even though default reasoning was intended to simulate the reasoning of typical human subjects, it turns out that following the operational semantics of default reasoning in a literal way leads to the peculiarities of autistic behavior observed in the literature.

Reinforcement learning algorithms can train an agent to operate successfully in a stationary environment. Most real-world environments, however, are subject to change over time. Research in the areas of transfer learning and lifelong learning addresses this problem by developing new algorithms that allow agents to adapt to environment change. Current trends in this area include model-free learning and data-driven adaptation methods. This paper explores in the opposite direction of those trends. Arguing that model-based algorithms may be better suited to the problem, it looks at adaptation in the context of model-based learning. Noting that standard algorithms themselves have some built-in capability for adaptation, it analyzes when and why a standard algorithm struggles to adapt to environment change. Then it experiments with lightweight and straightforward methods for adapting effectively.

Autonomous robots demand complex behavior to deal with unstructured environments. To meet these expectations, a robot needs to address a suite of problems associated with long term knowledge acquisition, representation, and execution in the presence of partial information. In this paper, we address these issues by the acquisition of broad, domain general skills using an intrinsically motivated reward function. We show how these skills can be represented compactly and used hierarchically to obtain complex manipulation skills. We further present a Bayesian model using the learned skills to model objects in the world, in terms of the actions they afford. We argue that our knowledge representation allows a robot to both predict the dynamics of objects in the world as well as recognize them.

In 2009, an average American spent 3 hours per day watching TV. Recent advances in TV entertainment technologies, such as on-demand content, browsing the Internet, and 3D displays, have changed the traditional role of the TV and turned it into the center of home entertainment. Most of these technologies are personal and would benefit from seamless identification of who sits in front of the TV. In this work, we propose a practical and highly accurate solution to this problem. This solution uses a camera, which is mounted on a TV, to recognize faces of people in front of the TV. To make the approach practical, we employ online learning on graphs and show that we can learn highly accurate face models in difficult circumstances from as little as one labeled example. To evaluate our solutions, we collected a 10-hour long dataset of 8 people who watch TV. Our precision and recall are in the upper nineties, and show the promise of utilizing our approach in an embedded setting.

Recently, information gain has been proposed as a candidate intrinsic motivation for lifelong learning agents that may not always have a specific task.  In the InfoMax control framework, reinforcement learning is used to find a control policy for a POMDP in which movement and sensing actions are selected to reduce Shannon entropy as quickly as possible. In this study, we implement InfoMax control on a robot which can move between objects and perform sound-producing manipulations on them.  We formulate a novel latent variable mixture model for acoustic similarities and learn InfoMax polices that allow the robot to rapidly reduce uncertainty about the categories of the objects in a room. We find that InfoMax with our improved acoustic model leads to policies which lead to high classification accuracy.  Interestingly, we also find that with an insufficient model, the InfoMax policy eventually learns to "bury its head in the sand" to avoid getting additional evidence that might increase uncertainty.  We discuss the implications of this finding for InfoMax as a principle of intrinsic motivation in lifelong learning agents.

Skill discovery algorithms in reinforcement learning typically identify single states or regions in state space that correspond to potential task-specific subgoals. However, such methods do not directly address the question of how many distinct skills are appropriate for solving the tasks that the agent faces. This can be highly inefficient when many identified subgoals correspond to the same underlying skill, but are all used in- dividually as skill goals. Furthermore, skills created in this manner are often only transferable to tasks that share iden- tical state spaces, since corresponding subgoals across tasks are not merged into a single skill goal. We show that these problems can be overcome by clustering subgoal data defined in an agent-space and using the resulting clusters as templates for skill termination conditions. Clustering via a Dirichlet process mixture model is used to discover a minimal, suffi- cient collection of portable skills.

The paper addresses the following issues:  (a) how to represent semantic information from natural language so that a vision model can utilize it?  (b) how to extract the salient textual information relevant to vision?  For a given domain, we present a new model of semantic extraction that takes into account word relatedness as well as word disambiguation in order to apply to a vision model. We automatically process the text transcripts and perform syntactic analysis to extract dependency relations. We then perform semantic extraction on the output to filter semantic entities related to actions. The resulting data are used to populate a matrix of co-occurrences utilized by the vision processing modules.  Results show that explicitly modeling the co-occurrence of actions and tools significantly improved performance.