Decentralized partially observable Markov decision processes (Dec-POMDPs) formalize the problem of designing individual controllers for a group of collaborative agents under stochastic dynamics and partial observability. Seeking a global optimum is difficult (NEXP complete), but seeking a Nash equilibrium -- each agent policy being a best response to the other agents -- is more accessible, and allowed addressing infinite-horizon problems with solutions in the form of finite state controllers. In this paper, we show that this approach can be adapted to cases where only a generative model (a simulator) of the Dec-POMDP is available. This requires relying on a simulation-based POMDP solver to construct an agent's FSC node by node. A related process is used to heuristically derive initial FSCs. Experiment with benchmarks shows that MC-JESP is competitive with exisiting Dec-POMDP solvers, even better than many offline methods using explicit models.

In human-robot collaboration, the objectives of the human are often unknown to the robot. Moreover, even assuming a known objective, the human behavior is also uncertain. In order to plan a robust robot behavior, a key preliminary question is then: How to derive realistic human behaviors given a known objective? A major issue is that such a human behavior should itself account for the robot behavior, otherwise collaboration cannot happen. In this paper, we rely on Markov decision models, representing the uncertainty over the human objective as a probability distribution over a finite set of objective functions (inducing a distribution over human behaviors). Based on this, we propose two contributions: 1) an approach to automatically generate an uncertain human behavior (a policy) for each given objective function while accounting for possible robot behaviors; and 2) a robot planning algorithm that is robust to the above-mentioned uncertainties and relies on solving a partially observable Markov decision process (POMDP) obtained by reasoning on a distribution over human behaviors. A co-working scenario allows conducting experiments and presenting qualitative and quantitative results to evaluate our approach.

This paper looks at solving collaborative planning problems formalized as Decentralized POMDPs (Dec-POMDPs) by searching for Nash equilibria, i.e., situations where each agent's policy is a best response to the other agents' (fixed) policies. While the Joint Equilibrium-based Search for Policies (JESP) algorithm does this in the finite-horizon setting relying on policy trees, we propose here to adapt it to infinite-horizon Dec-POMDPs by using finite state controller (FSC) policy representations. In this article, we (1) explain how to turn a Dec-POMDP with $N-1$ fixed FSCs into an infinite-horizon POMDP whose solution is an $N^\text{th}$ agent best response; (2) propose a JESP variant, called \infJESP, using this to solve infinite-horizon Dec-POMDPs; (3) introduce heuristic initializations for JESP aiming at leading to good solutions; and (4) conduct experiments on state-of-the-art benchmark problems to evaluate our approach.

We propose a Variational Time Series Feature Extractor (VTSFE), inspired by the VAE-DMP model of Chen et al., to be used for action recognition and prediction. Our method is based on variational autoencoders. It improves VAE-DMP in that it has a better noise inference model, a simpler transition model constraining the acceleration in the trajectories of the latent space, and a tighter lower bound for the variational inference. We apply the method for classification and prediction of whole-body movements on a dataset with 7 tasks and 10 demonstrations per task, recorded with a wearable motion capture suit. The comparison with VAE and VAE-DMP suggests the better performance of our method for feature extraction. An open-source software implementation of each method with TensorFlow is also provided. In addition, a more detailed version of this work can be found in the indicated code repository. Although it was meant to, the VTSFE hasn't been tested for action prediction, due to a lack of time in the context of Maxime Chaveroche's Master thesis at INRIA.

The paper describes ongoing integrated research on designing intelligent robots that can assist humans in making a situation assessment during Urban Search & Rescue (USAR) missions. These robots (rover, microcopter) are deployed during the early phases of an emergency response. The aim is to explore those areas of the disaster hotzone which are too dangerous or too difficult for a human to enter at that point. This requires the robots to be "intelligent" in the sense of being capable of various degrees of autonomy in acting and perceiving in the environment. At the same time, their intelligence needs to go beyond mere task-work. Robots and humans are interdependent. Human operators are dependent on these robots to provide information for a situation assessment. And robots are dependent on humans to help them operate (shared control) and perceive (shared assessment) in what are typically highly dynamic, largely unknown environments. Robots and humans need to form a team. The paper describes how various insights from robotics and Artificial Intelligence are combined, to develop new approaches for modeling human robot teaming. These approaches range from new forms of modeling situation awareness (to model distributed acting in dynamic space), human robot interaction (to model communication in teams), flexible planning (to model team coordination and joint action), and cognitive system design (to integrate different forms of functionality in a single system).

In this paper we examine the case of a mobile robot that is part of a human-robot urban search and rescue (USAR) team. During USAR scenarios, we would like the robot to have a geometrical-functional understand- ing of space, using which it can infer where to perform planned tasks in a manner that mimics human behav- ior. We assess the situation awareness of rescue work- ers during a simulated USAR scenario and use this as an empirical basis to build our robot’s spatial model. Based upon this spatial model, we present “functional map- ping” as an approach to identify regions in the USAR environment where planned tasks are likely to be opti- mally achievable. The system is deployed and evaluated in a simulated rescue scenario.

Earlier Teamwork implies communication with shared references work also evaluates the robustness of ICP against low constrained and symbols. The collaboration between robot and human is environments (Rusinkiewicz and Levoy 2001). This therefore highly dependent on a common representation of was mainly done in simulation so real word datasets targeting the environment. Part of this representation is a map, either this limitations could bring the analysis farther. An other global or local, that can serve both the robot to do its own problem, recently raised in vision registration (Mortensen, task and the human to increase his situation awareness, to Deng, and Shapiro 2005), is the problem of repetitive elements collaboratively plan and observe the evolution of a situation.

We present an original integration of high level planning and execution with incoming perceptual information from vision, SLAM, topological map segmentation and dialogue. The task of the robot system, implementing the integrated model, is to explore unknown areas and report detected objects to an operator, by speaking loudly. The knowledge base of the planner maintains a graph-based representation of the metric map that is dynamically constructed via an unsupervised topological segmentation method, and augmented with information about the type and position of detected objects, within the map, such as cars or containers. According to this knowledge the cognitive robot can infer strategies in so generating parametric plans that are instantiated from the perceptual processes. Finally, a model-based approach for the execution and control of the robot system is proposed to monitor, concurrently, the low level status of the system and the execution of the activities, in order to achieve the goal, instructed by the operator.