Although the use of metric fluents is fundamental to many practical planning problems, the study of heuristics to support fully automated planners working with these fluents remains relatively unexplored. The most widely used heuristic is the relaxation of metric fluents into interval-valued variables --- an idea first proposed a decade ago. Other heuristics depend on domain encodings that supply additional information about fluents, such as capacity constraints or other resource-related annotations. A particular challenge to these approaches is in handling interactions between metric fluents that represent exchange, such as the transformation of quantities of raw materials into quantities of processed goods, or trading of money for materials. The usual relaxation of metric fluents is often very poor in these situations, since it does not recognise that resources, once spent, are no longer available to be spent again. We present a heuristic for numeric planning problems building on the propositional relaxed planning graph, but using a mathematical program for numeric reasoning. We define a class of producer--consumer planning problems and demonstrate how the numeric constraints in these can be modelled in a mixed integer program (MIP). This MIP is then combined with a metric Relaxed Planning Graph (RPG) heuristic to produce an integrated hybrid heuristic. The MIP tracks resource use more accurately than the usual relaxation, but relaxes the ordering of actions, while the RPG captures the causal propositional aspects of the problem. We discuss how these two components interact to produce a single unified heuristic and go on to explore how further numeric features of planning problems can be integrated into the MIP. We show that encoding a limited subset of the propositional problem to augment the MIP can yield more accurate guidance, partly by exploiting structure such as propositional landmarks and propositional resources. Our results show that the use of this heuristic enhances scalability on problems where numeric resource interaction is key in finding a solution.

We address the problem faced by an autonomous agent that must achieve quick responses to a family of qualitatively-related tasks, such as a robot interacting with different types of human participants. We work in the setting where the tasks share a state-action space and have the same qualitative objective but differ in the dynamics and reward process. We adopt a transfer approach where the agent attempts to exploit common structure in learnt policies to accelerate learning in a new one. Our technique consists of a few key steps. First, we use a probabilistic model to describe the regions in state space which successful trajectories seem to prefer. Then, we extract policy fragments from previously-learnt policies for these regions as candidates for reuse. These fragments may be treated as options with corresponding domains and termination conditions extracted by unsupervised learning. Then, the set of reusable policies is used when learning novel tasks, and the process repeats. The utility of this method is demonstrated through experiments in the simulated soccer domain, where the variability comes from the different possible behaviours of opponent teams, and the agent needs to perform well against novel opponents.

The recently developed Efficient Lifelong Learning Algorithm (ELLA) acquires knowledge incrementally over a sequence of tasks, learning a repository of latent model components that are sparsely shared between models. ELLA shows strong performance in comparison to other multi-task learning algorithms, achieving nearly identical performance to batch multi-task learning methods while learning tasks sequentially in three orders of magnitude (over 1,000x) less time. In this paper, we evaluate several curriculum selection methods that allow ELLA to actively select the next task for learning in order to maximize performance on future learning tasks. Through experiments with three real and one synthetic data set, we demonstrate that active curriculum selection allows an agent to learn up to 50% more efficiently than when the agent has no control over the task order.

In many real-life design problems, there is a requirement to simultaneously balance multiple tasks or objectives in the system that are conflicting in nature, where minimizing one objective causes another to increase in value, thereby resulting in trade-offs between the objectives. For example, in embedded multi-core mobile devices and very large scale data centers, there is a continuous problem of simultaneously balancing interfering goals of maximal power savings and minimal performance delay with varying trade-off values for different application workloads executing on them. Typically, the optimal trade-offs for the executing workloads, lie on a difficult to determine optimal Pareto front. The nature of the problem requires learning over the lifetime of the mobile device or server with continuous evaluation and prediction of the trade-off settings on the system that balances the interfering objectives optimally. Towards this, we propose an on-line learning method, where the weights of experts for addressing the objectives are updated based on a convex combination of their relative performance in addressing all objectives simultaneously. An additional importance vector that assigns relative importance to each objective at every round is used, and is sampled from a convex cone pointed at the origin Our preliminary results show that the convex combination of the importance vector and the gradient of the potential functions of the learner's regret with respect to each objective ensure that in the next round, the drift (instantaneous regret vector), is the Pareto descent direction that enables better convergence to the optimal Pareto front.

Nowadays developing autonomous systems, which can act in various environments and interactively perform their assigned tasks, are intensively desirable. These systems would be ready to be applied in different fields such as medicine, controller robots and social life. Reinforcement learning is an attractive area of machine learning which addresses these concerns. In large scales, learning performance of an agent can be improved by using hierarchical Reinforcement Learning techniques and temporary extended actions. The higher level of abstraction helps the learning agent approach lifelong learning goals. In this paper a new method is presented for discovering subgoal states and constructing useful skills. The method utilizes Ant System optimization algorithm to identify bottleneck edges, which act like bridges between different connected areas of the problem space. Using discovered subgoals, the agent creates temporal abstractions, which enable it to explore more effectively. Experimental Results show that the proposed method can significantly improve the learning performance of the agent.

We present an integrated system for segmenting demonstrations, recognizing repeated skills, and generalizing multi-step tasks from unstructured demonstrations.  This method combines recent work in Bayesian nonparametric statistics and learning from demonstration with perception using an RGB-D camera to generalize a multi-step task on the PR2 mobile manipulator.  We demonstrate the potential of our framework to learn a large library of skills over time and discuss how it might be improved with additional integration of components such as active learning, interactive feedback from humans, and more advanced perception. 

In this report we address the role of trust in autonomous systems, and our progress in developing a theory of interdependence for the efficient control of hybrid teams and systems composed of robots, machines and humans working interchangeably. Sentient multi-agent systems require an aggregation process like data fusion. But conventional use of fusion for the control of UxV systems hinges on convergences to form patterns, increasing uncertainty. Present solutions appear to indicate stability for cooperative contexts and instability for competitive ones, in line with our theoretical expectations.

This paper reports a model of a developmental agent. It is inspired by some characteristics of Piagets sensorimotor stage. During this stage essential skills for creative thinking are developed. Our computational model attempts to shed some light about how these abilities arise and, in this way, contribute to the study of the developmental side of computational creativity.

In previous work we presented a computational framework that allows a robot or agent to reason about whether it should trust an interactive partner or whether the interactive partner trusts the robot  (Wagner & Arkin, 2011). This article examines the use of this framework in a well-known situation for examining trust--the Investor-Trustee game (King-Casas, Tomlin, Anen, Camerer, Quartz, & Montague, 2005). Our experiment pits the robot against a person in this game and explores the impact of recognizing and responding to trust signals. Our results demonstrate that the recognition that a person has intentionally placed themselves at risk allows the robot to reciprocate and, by doing so, improve both individuals play in the game. This work has implications for home healthcare, search and rescue, and military applications.

Emergency evacuations are dangerous situations for both evacuees and first responders. The use of automation in the form of guidance robots can reduce the danger to humans by both aiding evacuees and assisting first responders. This presents an interesting opportunity to explore the trust dynamic between frightened evacuees and automated robot guides. We present our work so far on designing robots to immediately generate trust as well as our initial concept of an algorithm for maintaining trust through interaction.