Efficient exploration is important for reinforcement learners to achieve high rewards. In multi-agent systems, coordinated exploration and behaviour is critical for agents to jointly achieve optimal outcomes. In this paper, we introduce a new general framework for improving coordination and performance of multi-agent reinforcement learners (MARL). Our framework, named Learnable Intrinsic-Reward Generation Selection algorithm (LIGS) introduces an adaptive learner, Generator that observes the agents and learns to construct intrinsic rewards online that coordinate the agents' joint exploration and joint behaviour. Using a novel combination of MARL and switching controls, LIGS determines the best states to learn to add intrinsic rewards which leads to a highly efficient learning process. LIGS can subdivide complex tasks making them easier to solve and enables systems of MARL agents to quickly solve environments with sparse rewards. LIGS can seamlessly adopt existing MARL algorithms and, our theory shows that it ensures convergence to policies that deliver higher system performance. We demonstrate its superior performance in challenging tasks in Foraging and StarCraft II.

This work explores the application of nonlinear oscillators coupled by electric field in water for collective tasks in underwater robotics. Such coupled oscillators operate in clear and colloidal (mud, bottom silt) water and represent a collective electrochemical sensor that is sensitive to global environmental parameters, geometry of common electric field and spatial dynamics of autonomous underwater vehicles (AUVs). Implemented in hardware and software, this approach can be used to create global awareness in the group of robots, which possess limited sensing and communication capabilities. Using oscillators from different AUVs enables extending the range limitations related to electric dipole of a single AUV. Applications of this technique are demonstrated for detecting the number of AUVs, distances between them, perception of dielectric objects, synchronization of behavior and discrimination between 'collective self' and 'collective non-self' through an 'electrical mirror'. These approaches have been implemented in several research projects with AUVs in fresh and salt water.

We study two state-of-the-art solutions to the multi-agent pickup and delivery (MAPD) problem based on different principles -- multi-agent path-finding (MAPF) and multi-agent reinforcement learning (MARL). Specifically, a recent MAPF algorithm called conflict-based search (CBS) and a current MARL algorithm called shared experience actor-critic (SEAC) are studied. While the performance of these algorithms is measured using quite different metrics in their separate lines of work, we aim to benchmark these two methods comprehensively in a simulated warehouse automation environment.

Abstract--We study cluster assignment in multi-agent networks. The process of reaching an agreement between agents is In this work we focus on homogeneous networks, that is one of the fundamental tasks for a multi-agent system (MAS). The problem we aim to solve is how to design graphs computation [1], robotics [2], biochemical systems [3], and that ensure the networked system will converge to a prescribed sensor networks [4]. A natural extension to the agreement cluster configuration, i.e., specifying the number of clusters problem is the cluster agreement problem, which seeks to and the number of agents within each cluster. Employing tools drive agents into groups. All the agents within the same group from group theory, we show that it is possible to design an should then reach an agreement.

The spontaneous formation and subsequent growth, dissolution, merger and competition of social groups bears similarities to physical phase transitions in metastable finite systems. We examine three different scenarios, percolation, spinodal decomposition and nucleation, to describe the formation of social groups of varying size and density. In our agent-based model, we use a feedback between the opinions of agents and their ability to establish links. Groups can restrict further link formation, but agents can also leave if costs exceed the group benefits. We identify the critical parameters for costs/benefits and social influence to obtain either one large group or the stable coexistence of several groups with different opinions. Analytic investigations allow to derive different critical densities that control the formation and coexistence of groups. Our novel approach sheds new light on the early stage of network growth and the emergence of large connected components.

This is the third article in our series on the personhood of autonomous systems. We followed this discussion by talking about Kant's concept of autonomy in the second article. Here, we will make an attempt to understand how autonomy is perceived in the computer science domain. You will often see individuals correlating autonomy with automation. However, both of these mechanisms can be performed separately without human interference.

We present a scalable tree search planning algorithm for large multi-agent sequential decision problems that require dynamic collaboration. Teams of agents need to coordinate decisions in many domains, but naive approaches fail due to the exponential growth of the joint action space with the number of agents. We circumvent this complexity through an approach that allows us to trade computation for approximation quality and dynamically coordinate actions. Our algorithm comprises three elements: online planning with Monte Carlo Tree Search (MCTS), factored representations of local agent interactions with coordination graphs, and the iterative Max-Plus method for joint action selection. We evaluate our approach on the benchmark SysAdmin domain with static coordination graphs and achieve comparable performance with much lower computation cost than our MCTS baselines. We also introduce a multi-drone delivery domain with dynamic coordination graphs, and demonstrate how our approach scales to large problems on this domain that are intractable for other MCTS methods.

This paper describes the heuristics used by the Shadoks team for the CG:SHOP 2021 challenge. This year's problem is to coordinate the motion of multiple robots in order to reach their targets without collisions and minimizing the makespan. It is a classical multi agent path finding problem with the specificity that the instances are highly dense in an unbounded grid. Using the heuristics outlined in this paper, our team won first place with the best solution to 202 out of 203 instances and optimal solutions to at least 105 of them. The main ingredients include several different strategies to compute initial solutions coupled with a heuristic called Conflict Optimizer to reduce the makespan of existing solutions.

Natural selection driven by interspecific and intraspecific competition is a fundamental evolutionary mechanism that has led to the wide diversity and complexity of species inhabiting Earth. The process is mirrored to a degree in contemporary AI research, where competitive multi-agent reinforcement learning (RL) environments have enabled machines to reach superhuman performance. Designing multi-agent RL environments with conditions conducive to the development of interesting and useful agent skills can however be a time-consuming and laborious process. A common approach in single-agent settings is domain randomization, where the agent is trained on a wide distribution of randomized environments. Recent works have improved this process via automatic environment curricula techniques that adapt environment distribution during training to maximize the number of environments that produce better and more robust skills.

Before going further, you'll need to understand the concept of game theory. Game theory is basically a branch of applied mathematics. In-game theories (How Game Theory Strategy Improves Decision Making), there are different available tools with the help of which different situations are analyzed. There are parties in-game theories mostly referred to as players and the decision they have taken are interdependent. This is a kind of playing chess in which the turn of one player is associated with the future strategy of the opponent player.