In decision-theoretic planning problems, such as (partially observable) Markov decision problems or coordination graphs, agents typically aim to optimize a scalar value function. However, in many real-world problems agents are faced with multiple possibly conflicting objectives. In such multi-objective problems, the value is a vector rather than a scalar, and we need methods that compute a coverage set, i.e., a set of solutions optimal for all possible trade-offs between the objectives. In this project propose new multi-objective planning methods that compute the so-called convex coverage set (CCS): the coverage set for when policies can be stochastic, or the preferences are linear. We show that the CCS has favorable mathematical properties, and is typically much easier to compute that the Pareto front, which is often axiomatically assumed as the solution set for multi-objective decision problems.

Perhaps surprisingly, it is possible to predict how long an algorithm will take to run on a previously unseen input, using machine learning techniques to build a model of the algorithm's runtime as a function of problem-specific instance features. Such models have many important applications and over the past decade, a wide variety of techniques have been studied for building such models. In this extended abstract of our 2014 AI Journal article of the same title, we summarize existing models and describe new model families and various extensions. In a comprehensive empirical analyis using 11 algorithms and 35 instance distributions spanning a wide range of hard combinatorial problems, we demonstrate that our new models yield substantially better runtime predictions than previous approaches in terms of their generalization to new problem instances, to new algorithms from a parameterized space, and to both simultaneously.

We introduce a framework that enables an agent to autonomously learn its own symbolic representation of a low-level, continuous environment. Propositional symbols are formalized as names for probability distributions, providing a natural means of dealing with uncertain representations and probabilistic plans. We determine the symbols that are sufficient for computing the probability with which a plan will succeed, and demonstrate the acquisition of a symbolic representation in a computer game domain.

This model is used to automatically synthetise a controller that maps executions to the next action to perform. Many heuristics for cost-optimal planning are The problem is thus cast as a synthesis problem from a based on linear programming. We cover several given specification. Two obstacles for this approach are that interesting heuristics of this type by a common a suitable model for the task is needed, and that the synthesis framework that fixes the objective function of the problem is intractable in general. But, this intractability does linear program. Within the framework, constraints not preclude the approach from being effective in meaningful from different heuristics can be combined in one cases. Planning is the model-based approach to autonomous heuristic estimate which dominates the maximum behaviour.

Monte-Carlo Tree Search (MCTS) algorithms such as UCT are an attractive online framework for solving planning under uncertainty problems modeled as a Markov Decision Process. However, MCTS search trees are constructed in flat state and action spaces, which can lead to poor policies for large problems. In a separate research thread, domain abstraction techniques compute symmetries to reduce the original MDP. This can lead to significant savings in computation, but these have been predominantly implemented for offline planning. This paper makes two contributions. First, we define the ASAP (Abstraction of State-Action Pairs) framework, which extends and unifies past work on domain abstractions by holistically aggregating both states and state-action pairs — ASAP uncovers a much larger number of symmetries in a given domain. Second, we propose ASAP-UCT, which implements ASAP-style abstractions within a UCT framework combining strengths of online planning with domain abstractions. Experimental evaluation on several benchmark domains shows up to 26% improvement in the quality of policies obtained over existing algorithms.

Capturing and exploiting structural knowledge of planning problems has shown to be a successful strategy for making the planning process more efficient. Plans can be decomposed into its constituent coherent subplans, called blocks, that encapsulate some effects and preconditions, reducing interference and thus allowing more deordering of plans. According to the nature of blocks, they can be straightforwardly transformed into useful macro-operators (shortly, macros). Macros are well known and widely studied kind of structural knowledge because they can be easily encoded in the domain model and thus exploited by standard planning engines. In this paper, we introduce a method, called BloMa, that learns domain-specific macros from plans, decomposed into macro-blocks which are extensions of blocks, utilising structural knowledge they capture. In contrast to existing macro learning techniques, macro-blocks are often able to capture high-level activities that form a basis for useful longer macros (i.e. those consisting of more original operators). Our method is evaluated by using the IPC benchmarks with state-of-the-art planning engines, and shows considerable improvement in many cases.

Whereas previous work on non-deterministic planning has focused on characterizing (and computing) "loopy" but "closed" plans, we look here at the kind of environments that these plans are to be executed in. In particular, we provide a logical characterization of the standard "fairness'' assumption used, and show that strong cyclic plans are correct solution concepts for fair environments.  We argue then that such logical characterization allows us to recast non-deterministic planning as a reactive synthesis task, and show that for a special case, recent efficient synthesis techniques can be applied.

Recent work in portfolios of problem solvers has shown their ability to outperform single-algorithm approaches in some tasks (e.g. SAT or Automated Planning). However, not much work has been devoted to a better understanding of the relationship between the order of the component solvers and the performance of the resulting portfolio over time. We propose to sort the component solvers in a sequential portfolio, such that the resulting ordered portfolio maximizes the probability of providing the largest performance at any point in time. We empirically show that our greedy approach efficiently obtains near-optimal performance over time. Also, it generalizes much better than an optimal approach which has been observed to suffer from overfitting.

Current motion planners, such as the ones available in ROS MoveIt, can solve difficult motion planning problems. However, these planners are not practical in unstructured, rapidly-changing environments. First, they assume that the environment is well-known, and static during planning and execution. Second, they do not support temporal constraints, which are often important for synchronization between a robot and other actors. Third, because many popular planners generate completely new trajectories for each planning problem, they do not allow for representing persistent control policy information associated with a trajectory across planning problems. We present Chekhov, a reactive, integrated motion planning and execution system that addresses these problems. Chekhov uses a Tube-based Roadmap in which the edges of the roadmap graph are families of trajectories called flow tubes, rather than the single trajectories commonly used in roadmap systems. Flow tubes contain control policy information about how to move through the tube, and also represent the dynamic limits of the system, which imply temporal constraints. This, combined with an incremental APSP algorithm for quickly finding paths in the roadmap graph, allows Chekhov to operate in rapidly changing environments. Testing in simulation, and with a robot testbed has shown improvement in planning speed and motion predictability over current motion planners.

Monte Carlo Tree Search (MCTS) is the state of the art algorithm for many games including the game of Go and General Game Playing (GGP). The standard algorithm for MCTS is Upper Confidence bounds applied to Trees (UCT). For games such as Go a big improvement over UCT is the Rapid Action Value Estimation (RAVE) heuristic. We propose to generalize the RAVE heuristic so as to have more accurate estimates near the leaves. We test the resulting algorithm named GRAVE for Atarigo, Knighthrough, Domineering and Go.