We consider the problem of dynamic self-reconfiguration in a modular self-reconfigurable robot (MSR). Previous MSR self-reconfiguration approaches search for new configurations only within the modules of the MSR that needs reconfiguration. In contrast, we describe a technique where an MSR that needs to reconfigure communicates with other MSRs in its vicinity to determine if modules can be shared from other MSRs, and then determines the best possible configuration among the combined set of modules. We model the MSR self-reconfiguration problem as a coalition structure generation problem within a coalition game theoretic framework. We formulate the coalition structure generation problem as a planning problem in the presence of uncertainty and propose an MDP-based algorithm to solve it. We have implemented our algorithm within an MSR called ModRED that is simulated on the Webots simulation platform. Our results show that using our self-reconfiguration algorithm, when an MSR needs to reconfigure, a new configuration that is within 5-7% of the globally optimal configuration can be determined. We have also shown that our algorithm performs comparably with another existing algorithm for determining optimal coalition structure.

Skill discovery algorithms in reinforcement learning typically identify single states or regions in state space that correspond to potential task-specific subgoals. However, such methods do not directly address the question of how many distinct skills are appropriate for solving the tasks that the agent faces. This can be highly inefficient when many identified subgoals correspond to the same underlying skill, but are all used in- dividually as skill goals. Furthermore, skills created in this manner are often only transferable to tasks that share iden- tical state spaces, since corresponding subgoals across tasks are not merged into a single skill goal. We show that these problems can be overcome by clustering subgoal data defined in an agent-space and using the resulting clusters as templates for skill termination conditions. Clustering via a Dirichlet process mixture model is used to discover a minimal, suffi- cient collection of portable skills.

Learning about users' preferences allows agents to make intelligent decisions on behalf of users. When we are eliciting preferences from a group of users, we can use the preferences of the users we have already processed to increase the efficiency of the elicitation process for the remaining users. However, current methods either require strong prior knowledge about the users' preferences or can be overly cautious and inefficient. Our method, based on standard techniques from non-parametric statistics, allows the controller to choose a balance between prior knowledge and efficiency. This balance is investigated through experimental results.

The growing use of autonomous agents in practice may require agents to cooperate as a team in situations where they have limited prior knowledge about one another, cannot communicate directly, or do not share the same world models. These situations raise the need to design ad hoc team members, i.e., agents that will be able to cooperate without coordination in order to reach an optimal team behavior. This paper considers problem of leading N-agent teams by a single agent toward their optimal joint utility, where the agents compute their next actions based only on their most recent observations of their teammates' actions. We show that compared to previous results in two-agent teams, in larger teams the agent might not be able to lead the team to the action with maximal joint utility. In these cases, the agent's optimal strategy leads the team to the best possible reachable cycle of joint actions. We describe a graphical model of the problem and a polynomial time algorithm for solving it. We then consider the problem of leading teams where the agents' base their actions on a longer history of past observations, showing that the an upper bound computation time exponential in the memory size is very likely to be tight.

Current methods for multi-AUV cooperation suffer in low communication environments. State of the art methods employ auctioneering or planning to determine a single AUV'task. These systems require communication to update models of teammates and tasks for efficient task selection. Most strategies assume a teammate is inoperable if a communication timeout is reached which reduces overall team efficiency. Including teammate prediction has been shown to mitigate efficiency degeneration due to low communication. However, there is no verification of a predicted teammate's task other than through eventual communication. A possible verification tool is behavior recognition. Current behavior recognition utilizes either overhead sensors or post mission analysis to track robot trajectories in order to infer their internal state. A system in which an AUV is capable of sensing a teammate, for example through a forward-looking sonar, and deducing it's behavior along with contextual information, such as location, will enable an AUV to determine that teammate's current task in the overall mission. This will allow for an accurate update of that teammate's model allowing the AUV to more efficiently determine its own next task rather than relying only on communication. This position paper posits that multi-AUV cooperation efficiency will improve in low communication environments with the combination of robust teammate prediction along with verification using behavior recognition.

The task of coordinating hundreds of mobile robots in one of Kiva System's warehouses presents many challenging multi-agent resource allocation problems. The resources include things like inventory, open orders, small shelving units, and the robots themselves. The types of resources can be classified by whether they are consumable, recycled, or scheduled. Further, the global optimization problem can be broken down into more manageable sub-problems, some of which map to (hard) versions of well known computational problems, but with a dynamic, temporal twist.

MapReduce provided a novel computing model for complex job decomposition and sub-tasks management to support cloud computing with large distributed data sets. However, its performance is significantly influenced by the working data distributions over those data sets. In this paper, we put forward a novel model to balance data distribution to improve cloud computing performance in data-intensive applications, such as distributed data mining. By extending the classic MapReduce model with an agent-aid layer and abstracting working load requests for data blocks as tokens, the agents can reason from previously received tokens about where to send other tokens in order to balance the working tasks and improve system performance. Our key contribution lies in building an efficient token routing algorithm in spite of agents' unknowing to the global state of data distribution in cloud. We also built a prototype of our system, and the experimental results show that our approach can significantly improve the efficiency of cloud computing.

Partially observable stochastic games (POSGs) are an attractive model for many multi-agent domains, but are computationally extremely difficult to solve. We present a new model, Markov games of incomplete information (MGII) which imposes a mild restriction on POSGs while overcoming their primary computational bottleneck. Finally we show how to convert a MGII into a continuous but bounded fully observable stochastic game. MGIIs represents the most general tractable model for multi-agent reinforcement learning to date.

We propose a mechanism to learn communicative action structure to analyze adversarial reasoning patterns in customer complaints. An efficient way to assist customers and companies is to reuse previous experience with similar agents. A formal representation of customer complaints and a machine learning technique for handling scenarios of interaction between conflicting human agents are proposed. It is shown that analyzing the structure of communicative actions without context information is frequently sufficient to advise on complaint resolution strategies. Therefore, being domain-independent, the proposed machine learning technique is a good complement to a wide range of customer response management applications where formal treatment of inter-human interactions is required.

Being able to compactly represent large state spaces is crucial in solving a vast majority of practical stochastic planning problems. This requirement is even more stringent in the context of multi-agent systems, in which the world to be modeled also includes the mental state of other agents. This leads to a hierarchy of beliefs that results in a continuous, unbounded set of possible interactive states, as in the case of Interactive POMDPs. In this paper, we describe a novel representation for interactive belief hierarchies that combines first-order logic and probability. The semantics of this new formalism is based on recursively partitioning the belief space at each level of the hierarchy; in particular, the partitions of the belief simplex at one level constitute the vertices of the simplex at the next higher level. Since in general a set of probabilistic statements only partially specifies a probability distribution over the space of interest, we adopt the maximum entropy principle in order to convert it to a full specification.