Network analysis is a growing field across many domains, including computer vision, social media marketing, transportation networks, and intelligence analysis. The growing use of digital communication devices and platforms, as well as persistent surveillance sensors, has resulted in explosion of the quantity of data and stretched the abilities of current technologies to process this data and draw meaningful conclusions. Current tools either require significant levels of manual intervention (e.g., to prepare the data, to define patterns, or to draw conclusions from data) or are unable to generalize to new data sources and analysis needs. In this paper, we present automated solutions to two major problems in network analysis: (a) finding patterns in the network data that contains high levels of noise and irrelevant information; and (b) learning repetitive patterns and dependencies between entities and attributes. Our modeling framework represents network data using multi-attributed graphs that can encode various discrete and continuous features and relationships between network entities. The pattern search and learning model is based on probabilistic multi-attributed graph matching, and implemented using distributed message passing algorithms. Our algorithms achieved high accuracy rates in learning and finding patterns in the data, are flexible to new domains and data types, and scale to large datasets using the Map-Reduce framework.

In the real world we observe a proliferation of regional dialects and jargons. Most of the research on naming conventions focuses on how to explain the process that allows a single naming convention to establish itself. This paper presents a different approach that aims to investigate why different conventions may emerge and coexist for a certain amount of time. The naming game is an abstraction of lexical acquisition dynamics, in which n agents try to find an agreement on the names to give to objects. To understand how different heterogeneous conventions emerge, I discuss a naming game model that takes into account experimental data on human and animal learning.

There is growing interest in automating the detection of interesting new developments in science and technology. BAE Systems is pursuing ARBITER (Abductive Reasoning Based on Indicators and Topics of EmeRgence), a multi-disciplinary study and development effort to analyze full- text and metadata for indicators of emergent technologies and scientific fields. To define these indicators, our team has applied the primary insights of actant network theory developed within the disciplines of Science and Technology Studies and the history of technology and science to create a pragmatic theory of technoscientific emergence. Specifically, this practical theory articulates emergence in terms of the robustness of actant networks. This applied actant-network theory currently guides our definition of indicators and indicator patterns for the ARBITER system, and represents a novel contribution to the discussion of emergent technologies and fields. Several elements of our theory were validated with 15 case studies and 25 example technologies.

We consider the problem of extending manually trainedagents via evaluative reinforcement (TAMER) in con-tinuous state and action spaces. The early work TAMERframework allows a non-technical human to train anagent through a natural form of human feedback, neg-ative or positive. The advantages of TAMER havebeen shown on applications such as training Tetris andMountain Car with only human feedback, Cart-poleand Mountain Car with human feedback and environ-ment reward (augmenting reinforcement learning withhuman feedback). However, those methods are origi-nally designed for discrete state-action, or continuousstate-discrete action problems. In this paper, we intro-duce an extension of TAMER to allow both continu-ous states and actions. The new scheme, actor-criticTAMER, extends the original TAMER to allow usingany general function approximation of a human trainer’sreinforcement signal. Our extension still allows rein-forcement learning to be easily combined with humanfeedback. The experimental results show that the pro-posed method helps a human trainer successfully trainan agent in two continuous state-action domains: Moun-tain Car, and Cart-pole (balancing).

Most learning from demonstration algorithms are implemented with a certain set of variables that are known to be important for the agent. The agent is hardcoded to use those variables for learning the task (or a set of parameters). In this work we try to understand the causal structure of a demonstrated task in order to find: which variables cause what other variables to change, and which variables are independent from the others. We used a realistic simulator to record a simple pick and place task demonstration data, and recovered different causal models using the data in Tetrad, a computer program that searches for causal and statistical models. Our findings show that it is possible to deduce irrelevant variables to a demonstrated task, using the recovered causal structure.

Members of a multi-robot team, operating within close quarters, need to avoid crashing into each other. Simple collision avoidance methods can be used to prevent such collisions, typically by computing the distance to other robots and stopping, perhaps moving away, when this distance falls below a certain threshold. While this approach may avoid disaster, it may also reduce the team's efficiency if robots halt for a long time to let others pass by or if they travel further to move around one another. This paper reports on experiments where a human operator, through a graphical user interface, watches robots perform an exploration task. The operator can manually suspend robots' movements before they crash into each other, and then resume their movements when their paths are clear. Experiment logs record the robots' states when they are paused and resumed. A behavior pattern for collision avoidance is learned, by classifying the states of the robots' environment when the human operator issues "wait" and "resume" commands. Preliminary results indicate that it is possible to learn a classifier which models these behavior patterns, and that different human operators consider different factors when making decisions about stopping and starting robots.

Intelligent systems promise to amplify, augment, and extend innate human abilities. A principal example is that of assistive rehabilitation robots---artificial intelligence and machine learning enable new electromechanical systems that restore biological functions lost through injury or illness. In order for an intelligent machine to assist a human user, it must be possible for a human to communicate their intentions and preferences to their non-human counterpart. While there are a number of techniques that a human can use to direct a machine learning system, most research to date has focused on the contrasting strategies of instruction and reward. The primary contribution of our work is to demonstrate that the middle ground between instruction and reward is a fertile space for research and immediate technological progress. To support this idea, we introduce the setting of human-prompted switching, and illustrate the successful combination of switching with interactive learning using a concrete real-world example: human control of a multi-joint robot arm. We believe techniques that fall between the domains of instruction and reward are complementary to existing approaches, and will open up new lines of rapid progress for interactive human training of machine learning systems.

Learning from demonstration (LfD) is a promising technique for instructing/teaching autonomous systems based on demonstrations from people who may have little to no experience with robots. An important aspect to LfD is the communication method used to transfer knowledge from an instructor to a robot. The communication method affects the complexity of the demonstration process for instructors, the range of tasks a robot can learn, and the learning algorithm itself. We have designed a graphical interface and an instructional language to provide an intuitive teaching system. The drawback to simplifying the teaching interface is that the resulting demonstration data are less structured, adding complexity to the learning process. This additional complexity is handled through the combination of a minimal set of predefined behaviors and a task representation capable of learning probabilistic policies over a set of behaviors. The predefined behaviors consist of finite actions a robot can perform, which act as building blocks for more complex tasks.

In standard passive imitation learning, the goal is to learn an expert’s policy by passively observing full execution trajectories of it. Unfortunately, generating such trajectories can require substantial expert effort and be impractical in some cases. In this paper, we consider Active Imitation Learning (AIL) with the goal of reducing this effort by querying the expert about the desired action at individual states, which are selected based on answers to past queries and the learner’s interactions with an environment simulator. Our new approach is based on reducing AIL to i.i.d. active learning, which can leverage progress in the i.i.d. setting. We introduce and analyze reductions for both non-stationary and stationary policies, showing that the label complexity (number of queries) of AIL can be substantially less than passive learning. We also introduce a practical algorithm inspired by the reductions, which is shown to be highly effective in four test domains compared to a number of alternatives.

We present an approach for enabling in-home service robots to follow natural language commands from non-expert users, with a particular focus on spatial language understanding. Specifically, we propose an extension to the semantic field model of spatial prepositions that enables the representation of dynamic spatial relations involving paths. The relevance of the proposed methodology to interactive robot learning is discussed, and the paper concludes with a description of how we plan to integrate and evaluate our proposed model with end-users.