Almost all multi-agent reinforcement learning algorithms without communication follow the principle of centralized training with decentralized execution. During centralized training, agents can be guided by the same signals, such as the global state. During decentralized execution, however, agents lack the shared signal. Inspired by viewpoint invariance and contrastive learning, we propose consensus learning for cooperative multi-agent reinforcement learning in this paper. Although based on local observations, different agents can infer the same consensus in discrete space. During decentralized execution, we feed the inferred consensus as an explicit input to the network of agents, thereby developing their spirit of cooperation. Our proposed method can be extended to various multi-agent reinforcement learning algorithms with small model changes. Moreover, we carry out them on some fully cooperative tasks and get convincing results.

This article considers consensus problem of multiagent systems with double integrator dynamics under nonuniform sampling. It is considered the maximum sampling time can be selected arbitrarily. Moreover, the communication graph can change to any possible topology as long as its associated graph Laplacian has eigenvalues in a given region, which can be selected arbitrarily. Existence of a controller that ensures consensus in this setting is shown when the changing topology graphs are balanced and has a spanning tree. Also, explicit bounds for controller parameters are given. A novel sufficient condition is given to solve the consensus problem based on making the closed loop system matrix a contraction using a particular coordinate system for general linear dynamics. It is shown that the given condition immediately generalizes to changing topology in the case of balanced topology graphs. This condition is applied to double integrator dynamics to obtain explicit bounds on the controller.

Development of guidance, navigation and control frameworks/algorithms for swarms attracted significant attention in recent years. That being said, algorithms for planning swarm allocations/trajectories for engaging with enemy swarms is largely an understudied problem. Although small-scale scenarios can be addressed with tools from differential game theory, existing approaches fail to scale for large-scale multi-agent pursuit evasion (PE) scenarios. In this work, we propose a reinforcement learning (RL) based framework to decompose to large-scale swarm engagement problems into a number of independent multi-agent pursuit-evasion games. We simulate a variety of multi-agent PE scenarios, where finite time capture is guaranteed under certain conditions. The calculated PE statistics are provided as a reward signal to the high level allocation layer, which uses an RL algorithm to allocate controlled swarm units to eliminate enemy swarm units with maximum efficiency. We verify our approach in large-scale swarm-to-swarm engagement simulations.

The Age-of-Information (AoI) metric has been widely studied in the theoretical communication networks and queuing systems literature. However, experimental evaluation of its applicability to complex real-world time-sensitive systems is largely lacking. In this work, we develop, implement, and evaluate an AoI-based application layer middleware that enables the customization of WiFi networks to the needs of time-sensitive applications. By controlling the storage and flow of information in the underlying WiFi network, our middleware can: (i) prevent packet collisions; (ii) discard stale packets that are no longer useful; and (iii) dynamically prioritize the transmission of the most relevant information. To demonstrate the benefits of our middleware, we implement a mobility tracking application using a swarm of UAVs communicating with a central controller via WiFi. Our experimental results show that, when compared to WiFi-UDP/WiFi-TCP, the middleware can improve information freshness by a factor of 109x/48x and tracking accuracy by a factor of 4x/6x, respectively. Most importantly, our results also show that the performance gains of our approach increase as the system scales and/or the traffic load increases.

In the Approval Participatory Budgeting problem an agent prefers a set of projects $W'$ over $W$ if she approves strictly more projects in $W'$. A set of projects $W$ is in the core, if there is no other set of projects $W'$ and set of agents $K$ that both prefer $W'$ over $W$ and can fund $W'$. It is an open problem whether the core can be empty, even when project costs are uniform. the latter case is known as the multiwinner voting core. We show that in any instance with uniform costs or with at most four projects (and any number of agents), the core is nonempty.

In this paper, we consider the partial gathering problem of mobile agents in synchronous dynamic bidirectional ring networks. When k agents are distributed in the network, the partial gathering problem requires, for a given positive integer g (< k), that agents terminate in a configuration such that either at least g agents or no agent exists at each node. So far, the partial gathering problem has been considered in static graphs. In this paper, we start considering partial gathering in dynamic graphs. As a first step, we consider this problem in 1-interval connected rings, that is, one of the links in a ring may be missing at each time step. In such networks, focusing on the relationship between the values of k and g, we fully characterize the solvability of the partial gathering problem and analyze the move complexity of the proposed algorithms when the problem can be solved. First, we show that the g-partial gathering problem is unsolvable when k <= 2g. Second, we show that the problem can be solved with O(n log g) time and the total number of O(gn log g) moves when 2g + 1 <= k <= 3g - 2. Third, we show that the problem can be solved with O(n) time and the total number of O(kn) moves when 3g - 1 <= k <= 8g - 4. Notice that since k = O(g) holds when 3g - 1 <= k <= 8g - 4, the move complexity O(kn) in this case can be represented also as O(gn). Finally, we show that the problem can be solved with O(n) time and the total number of O(gn) moves when k >= 8g - 3. These results mean that the partial gathering problem can be solved also in dynamic rings when k >= 2g + 1. In addition, agents require a total number of \Omega(gn) moves to solve the partial (resp., total) gathering problem. Thus, when k >= 3g - 1, agents can solve the partial gathering problem with the asymptotically optimal total number of O(gn) moves.

This work addresses the problem of sharing partial information within social learning strategies. In traditional social learning, agents solve a distributed multiple hypothesis testing problem by performing two operations at each instant: first, agents incorporate information from private observations to form their beliefs over a set of hypotheses; second, agents combine the entirety of their beliefs locally among neighbors. Within a sufficiently informative environment and as long as the connectivity of the network allows information to diffuse across agents, these algorithms enable agents to learn the true hypothesis. Instead of sharing the entirety of their beliefs, this work considers the case in which agents will only share their beliefs regarding one hypothesis of interest, with the purpose of evaluating its validity, and draws conditions under which this policy does not affect truth learning. We propose two approaches for sharing partial information, depending on whether agents behave in a self-aware manner or not. The results show how different learning regimes arise, depending on the approach employed and on the inherent characteristics of the inference problem. Furthermore, the analysis interestingly points to the possibility of deceiving the network, as long as the evaluated hypothesis of interest is close enough to the truth.

Recently, model-based agents have achieved better performance than model-free ones using the same computational budget and training time in single-agent environments. However, due to the complexity of multi-agent systems, it is tough to learn the model of the environment. The significant compounding error may hinder the learning process when model-based methods are applied to multi-agent tasks. This paper proposes an implicit model-based multi-agent reinforcement learning method based on value decomposition methods. Under this method, agents can interact with the learned virtual environment and evaluate the current state value according to imagined future states in the latent space, making agents have the foresight. Our approach can be applied to any multi-agent value decomposition method. The experimental results show that our method improves the sample efficiency in different partially observable Markov decision process domains.

From Deep Blue's victory over chess grandmaster Garry Kasparov to AlphaGo being the first computer program to defeat a Go World Champion, unbeatable superhuman agents have paved a new path for remarkable advancements made in AI. However, the primary question remains whether AI can create agents that can use language to negotiate and collaborate with others to achieve strategic goals in a manner comparable to humans. As it involves players mastering the art of understanding other people's perspectives and devising methods appropriately to persuade them to make agreements and form alliances with others, Diplomacy has long been considered a near-impossible challenge in AI. The complexity of human emotions makes it simple to learn these diplomatic skills. Nevertheless, the question remains: can artificially intelligent machines achieve this level of understanding and persuasion skills?

Motion prediction is highly relevant to the perception of dynamic objects and static map elements in the scenarios of autonomous driving. In this work, we propose PIP, the first end-to-end Transformer-based framework which jointly and interactively performs online mapping, object detection and motion prediction. PIP leverages map queries, agent queries and mode queries to encode the instance-wise information of map elements, agents and motion intentions, respectively. Based on the unified query representation, a differentiable multi-task interaction scheme is proposed to exploit the correlation between perception and prediction. Even without human-annotated HD map or agent's historical tracking trajectory as guidance information, PIP realizes end-to-end multi-agent motion prediction and achieves better performance than tracking-based and HD-map-based methods. PIP provides comprehensive high-level information of the driving scene (vectorized static map and dynamic objects with motion information), and contributes to the downstream planning and control. Code and models will be released for facilitating further research.