This paper presents a new Monte-Carlo search algorithm for very large sequential decision-making problems. We apply non-linear regression within Monte-Carlo search, online, to estimate a state-action value function from the outcomes of random roll-outs. This value function generalizes between related states and actions, and can therefore provide more accurate evaluations after fewer rollouts. A further significant advantage of this approach is its ability to automatically extract and leverage domain knowledge from external sources such as game manuals. We apply our algorithm to the game of Civilization II, a challenging multi-agent strategy game with an enormous state space and around 10^21 joint actions. We approximate the value function by a neural network, augmented by linguistic knowledge that is extracted automatically from the official game manual. We show that this non-linear value function is significantly more efficient than a linear value function, which is itself more efficient than Monte-Carlo tree search. Our non-linear Monte-Carlo search wins over 78% of games against the built-in AI of Civilization II.

Object detection is a basic skill for a robot to perform tasks in human environments. In order to build a good object classifier, a large training set of labeled images is required; this is typically collected and labeled (often painstakingly) by a human. This method is not scalable and therefore limits the robot's detection performance. We propose an algorithm for a robot to collect more data in the environment during its training phase so that in the future it could detect objects more reliably. The first step is to plan a path for collecting additional training images, which is hard because a previously visited location affects the decision for the future locations. One key component of our work is path planning by building a sparse graph that captures these dependencies. The other key component is our learning algorithm that weighs the errors made in robot's data collection process while updating the classifier. In our experiments, we show that our algorithms enable the robot to improve its object classifiers significantly.

In this paper we describe a software assistant agent that can proactively assist human users situated in a time-constrained environment to perform normative reasoning--reasoning about prohibitions and obligations--so that the user can focus on her planning objectives. In order to provide proactive assistance, the agent must be able to 1) recognize the user's planned activities, 2) reason about potential needs of assistance associated with those predicted activities, and 3) plan to provide appropriate assistance suitable for newly identified user needs. To address these specific requirements, we develop an agent architecture that integrates user intention recognition, normative reasoning over a user's intention, and planning, execution and replanning for assistive actions. This paper presents the agent architecture and discusses practical applications of this approach.

This requires, for Goal-driven autonomy (GDA) is a reflective model example, experts to anticipate what discrepancies can occur, of goal reasoning that controls the focus of an identify what goals can be formulated, and define their agent's planning activities by dynamically relative priority. However, few techniques have been resolving unexpected discrepancies in the world investigated for learning this knowledge, and those that do state, which frequently arise when solving tasks in learn only goal formulation knowledge (Weber et al. 2010; complex environments. GDA agents have Powell et al. 2011). This can be problematic; while these performed well on such tasks by integrating agents may perform well in simple environments, in others a methods for discrepancy recognition, explanation, domain expert might not know the (state) expectations for goal formulation, and goal management. However, executing every action in every state, nor which goal should they require substantial domain knowledge, be pursued to resolve every possible discrepancy, or even including what constitutes a discrepancy and how the space of all possible discrepancies.

Replanning via determinization is a recent, popular approach for onlineplanning in MDPs. In this paper we adapt this idea to classical,non-stochastic domains with partial information and sensing actions. At eachstep we generate a candidate plan which solves a classical planning probleminduced by the original problem. We execute this plan as long as it is safeto do so. When this is no longer the case, we replan.The classical planning problem we generate is based on the $T_0$ translation, in which the classical state captures the knowledge state of theagent. We overcome the non-determinism in sensing actions, and the large domain size introduced by $T_0$ by using state sampling. Our planner also employs a novel, lazy, regression-based method for querying the belief state.

This paper presents a plan recognition algorithm for inferring student behavior using virtual science laboratories. The algorithm extends existing plan recognition technology and was integrated with an existing educational application for chemistry. Automatic recognition of students’ activities in virtual laboratories can provide important information to teachers as well as serve as the basis for intelligent tutoring. Student use of virtual laboratories presents several challenges: Students may repeat activities indefinitely, interleave between activities, and engage in exploratory behavior using trial-anderror. The plan recognition algorithm uses a recursive grammar that heuristically generates plans on the fly, taking into account chemical reactions and effects to determine students’ intended high-level actions. The algorithm was evaluated empirically on data obtained from college students using virtual laboratory software for teaching chemistry. Results show that the algorithm was able to (1) infer the plans used by students to construct their models; (2) recognize such key processes as titration and dilution when they occurred in students’ work; (3) identify partial solutions; (4) isolate sequences of actions that were part of a single error.

Plan recognition is the problem of inferring the goals and plans of an agent from partial observations of her behavior. Recently, it has been shown that the problem can be formulated and solved using planners, reducing plan recognition to plan generation. In this work, we extend this model-based approach to plan recognition to the POMDP setting, where actions are stochastic and states are partially observable. The task is to infer a probability distribution over the possible goals of an agent whose behavior results from a POMDP model. The POMDP model is shared between agent and observer except for the true goal of the agent that is hidden to the observer. The observations are action sequences O that may contain gaps as some or even most of the actions done by the agent may not be observed. We show that the posterior goal distribution P ( G | O ) can be computed from the value function V G ( b ) over beliefs b  generated by the POMDP planner for each possible goal G. Some extensions of the basic framework are discussed, and a number of experiments are reported.

My work focuses on building computational agents that assist people in managing their activities in environments in which tempo and complexity outstrip people’s cognitive capacity,such as in coordinating rescue teams in the aftermath of a disaster, or in helping people with dementia manage their everyday lives. A critical challenge faced in such environments is not only that individuals must factor complicated constraints into deciding how and when to act on their own goals, but also that their decisions are further constrained by choices made by others with whom they interact, such as between cooperating teams in disaster relief or between patients and caregivers in an assisted-living facility. An additional challenge in such situations is that the interests of individuals, such as privacy and autonomy, along with slow, costly, uncertain,or otherwise problematic communication may further limitindividuals’ abilities to work together. My work assumes that a computational agent is associated with each individual, and that these agents will work together efficiently to manage individual and joint activities, while maintaining autonomy and privacy to the extent possible.

Partial-order plans (POPs) have the capacity to compactly represent numerous distinct plan linearizations and as a consequence are inherently robust. We exploit this robustness to do effective execution monitoring. We characterize the conditions under which a POP remains viable as the regression of the goal through the structure of a POP. We then develop a method for POP execution monitoring via a structured policy, expressed as an ordered algebraic decision diagram. The policy encompasses both state evaluation and action selection, enabling an agent to seamlessly switch between POP linearizations to accommodate unexpected changes during execution. We demonstrate the effectiveness of our approach by comparing it empirically and analytically to a standard technique for execution monitoring of sequential plans. On standard benchmark planning domains, our approach is 2 to 17 times faster and up to 2.5 times more robust than comparable monitoring of a sequential plan. On POPs that have few ordering constraints among actions, our approach is significantly more robust, with the ability to continue executing in up to an exponential number of additional states.

There is a huge and growing number of systems that depend on batteries for power supply, ranging from small mobile devices to large high-powered systems such as electrical substations. In most of these systems, there are significant user-benefits or engineering reasons to base the supply on multiple batteries, with load being switched between batteries by a control system. The key to efficient use of multiple batteries lies in the design of effective policies for the management of the switching of load between them. This paper describes work in which we show that automated planning can produce much more effective policies than other approaches to multiple battery load management in the literature.