In this paper, we consider the multi-robot path execution problem where a group of robots move on predefined paths from their initial to target positions while avoiding collisions and deadlocks in the face of asynchrony. We first show that this problem can be reformulated as a distributed resource allocation problem and, in particular, as an instance of the well-known Drinking Philosophers Problem (DrPP). By careful construction of the drinking sessions capturing shared resources, we show that any existing solutions to DrPP can be used to design robot control policies that are collectively collision and deadlock-free. We then propose modifications to an existing DrPP algorithm to allow more concurrent behavior, and provide conditions under which our method is deadlock-free. Our method does not require robots to know or to estimate the speed profiles of other robots and results in distributed control policies. We demonstrate the efficacy of our method on simulation examples, which show competitive performance against the state-of-the-art.

Overall, we explore the possibility of emergent behaviors in a multi-agent setting with a simple goal of predicting the next state of the game. By giving each cell agency with a convolutional neural network, we were able to explore more complex multi-agent behavior, giving each agent a short term memory in the form of convolutional parameters, and the ability to explore vs exploit their environment through their output movements. The goal was to explore the types of emerging behavior from groups of CNN agents with a selective pressure toward better predictions of the next state. Question In an environment where pixel agents are given mechanisms to self-replicate, compete, communicate, and predict, are these channels enough for the emergence of centralized control from distinct agents? Related Work This work was heavily inspired by the paper "Growing Neural Cellular Automata" 1, as well as by the original Game of Life by John Conway 2. The ResNet architecture was also borrowed and modified from this tutorial 3. Finally, the idea of a fitness value for the cells comes from the field of genetic algorithms. The guiding question and following philosophical implications are deeply connected to the works of Deleuze and Simondon among other philosophers and physicists. Assumptions Macroscopic wholes can emerge from discretized atomic agents Intelligent, accurate predictions of the world emerge from resource scarcity, thus a system requiring accurate predictions to grow imposes resource constraints correlated to those which life imposes on cell division and growth. We can model primordial selves with self replicating simplified agents given arbitrary, nonstationary bounds on themselves in the form of a pixel.

Reinforcement Learning (RL) is currently one of the most commonly used techniques for traffic signal control (TSC), which can adaptively adjusted traffic signal phase and duration according to real-time traffic data. However, a fully centralized RL approach is beset with difficulties in a multi-network scenario because of exponential growth in state-action space with increasing intersections. Multi-agent reinforcement learning (MARL) can overcome the high-dimension problem by employing the global control of each local RL agent, but it also brings new challenges, such as the failure of convergence caused by the non-stationary Markov Decision Process (MDP). In this paper, we introduce an off-policy nash deep Q-Network (OPNDQN) algorithm, which mitigates the weakness of both fully centralized and MARL approaches. The OPNDQN algorithm solves the problem that traditional algorithms cannot be used in large state-action space traffic models by utilizing a fictitious game approach at each iteration to find the nash equilibrium among neighboring intersections, from which no intersection has incentive to unilaterally deviate. One of main advantages of OPNDQN is to mitigate the non-stationarity of multi-agent Markov process because it considers the mutual influence among neighboring intersections by sharing their actions. On the other hand, for training a large traffic network, the convergence rate of OPNDQN is higher than that of existing MARL approaches because it does not incorporate all state information of each agent. We conduct an extensive experiments by using Simulation of Urban MObility simulator (SUMO), and show the dominant superiority of OPNDQN over several existing MARL approaches in terms of average queue length, episode training reward and average waiting time.

Multi-agent settings remain a fundamental challenge in the reinforcement learning (RL) domain due to the partial observability and the lack of accurate real-time interactions across agents. In this paper, we propose a new method based on local communication learning to tackle the multi-agent RL (MARL) challenge within a large number of agents coexisting. First, we design a new communication protocol that exploits the ability of depthwise convolution to efficiently extract local relations and learn local communication between neighboring agents. To facilitate multi-agent coordination, we explicitly learn the effect of joint actions by taking the policies of neighboring agents as inputs. Second, we introduce the mean-field approximation into our method to reduce the scale of agent interactions. To more effectively coordinate behaviors of neighboring agents, we enhance the mean-field approximation by a supervised policy rectification network (PRN) for rectifying real-time agent interactions and by a learnable compensation term for correcting the approximation bias. The proposed method enables efficient coordination as well as outperforms several baseline approaches on the adaptive traffic signal control (ATSC) task and the StarCraft II multi-agent challenge (SMAC).

The decarbonization of buildings presents new challenges for the reliability of the electrical grid as a result of the intermittency of renewable energy sources and increase in grid load brought about by end-use electrification. To restore reliability, grid-interactive efficient buildings can provide flexibility services to the grid through demand response. Residential demand response programs are hindered by the need for manual intervention by customers. To maximize the energy flexibility potential of residential buildings, an advanced control architecture is needed. Reinforcement learning is well-suited for the control of flexible resources as it is able to adapt to unique building characteristics compared to expert systems. Yet, factors hindering the adoption of RL in real-world applications include its large data requirements for training, control security and generalizability. Here we address these challenges by proposing the MERLIN framework and using a digital twin of a real-world 17-building grid-interactive residential community in CityLearn. We show that 1) independent RL-controllers for batteries improve building and district level KPIs compared to a reference RBC by tailoring their policies to individual buildings, 2) despite unique occupant behaviours, transferring the RL policy of any one of the buildings to other buildings provides comparable performance while reducing the cost of training, 3) training RL-controllers on limited temporal data that does not capture full seasonality in occupant behaviour has little effect on performance. Although, the zero-net-energy (ZNE) condition of the buildings could be maintained or worsened as a result of controlled batteries, KPIs that are typically improved by ZNE condition (electricity price and carbon emissions) are further improved when the batteries are managed by an advanced controller.

Resilience denotes the capacity of a system to withstand shocks and its ability to recover from them. We develop a framework to quantify the resilience of highly volatile, non-equilibrium social organizations, such as collectives or collaborating teams. It consists of four steps: (i) \emph{delimitation}, i.e., narrowing down the target systems, (ii) \emph{conceptualization}, .e., identifying how to approach social organizations, (iii) formal \emph{representation} using a combination of agent-based and network models, (iv) \emph{operationalization}, i.e. specifying measures and demonstrating how they enter the calculation of resilience. Our framework quantifies two dimensions of resilience, the \emph{robustness} of social organizations and their \emph{adaptivity}, and combines them in a novel resilience measure. It allows monitoring resilience instantaneously using longitudinal data instead of an ex-post evaluation.

Multi-agent reinforcement learning (MARL) recently has achieved tremendous success in a wide range of fields. However, with a black-box neural network architecture, existing MARL methods make decisions in an opaque fashion that hinders humans from understanding the learned knowledge and how input observations influence decisions. Our solution is MIXing Recurrent soft decision Trees (MIXRTs), a novel interpretable architecture that can represent explicit decision processes via the root-to-leaf path of decision trees. We introduce a novel recurrent structure in soft decision trees to address partial observability, and estimate joint action values via linearly mixing outputs of recurrent trees based on local observations only. Theoretical analysis shows that MIXRTs guarantees the structural constraint with additivity and monotonicity in factorization. We evaluate MIXRTs on a range of challenging StarCraft II tasks. Experimental results show that our interpretable learning framework obtains competitive performance compared to widely investigated baselines, and delivers more straightforward explanations and domain knowledge of the decision processes.

It's been another interesting year in the world of artificial intelligence. We've seen large language models grow even larger, conferences returning to physical events, a raft of new policy developments, and machine learning techniques applied across the arts. Buckle up and join us for the ride as we review the year just gone. Research into both fundamental and applied aspects of artificial intelligence and machine learning continues apace. Yue Ma and colleagues used machine-learning techniques to identify antimicrobial peptides encoded by the genome sequences of microbes in the human gut.

Sarvapali (Gopal) Ramchurn is a Professor of Artificial Intelligence, Turing Fellow, and Fellow of the Institution of Engineering and Technology. He is the Director of the UKRI Trustworthy Autonomous Systems hub and Co-Director of the Shell-Southampton Centre for Maritime Futures. He is also a Co-CEO of Empati Ltd, an AI startup working on decentralised green hydrogen technologies. His research is about the design of Responsible Artificial Intelligence for socio-technical applications including energy systems and disaster management. Ferdinando Rodriguez y Baena is Professor of Medical Robotics in the Department of Mechanical Engineering at Imperial College, where he leads the Mechatronics in Medicine Laboratory and the Applied Mechanics Division. He has been the Engineering Co-Director of the Hamlyn Centre, which is part of the Institute of Global Health Innovation, since July 2020.

Given the wealth inequality worldwide, there is an urgent need to identify the mode of wealth exchange through which it arises. To address the research gap regarding models that combine equivalent exchange and redistribution, this study compares an equivalent market exchange with redistribution based on power centers and a nonequivalent exchange with mutual aid using the Polanyi, Graeber, and Karatani modes of exchange. Two new exchange models based on multi-agent interactions are reconstructed following an econophysics approach for evaluating the Gini index (inequality) and total exchange (economic flow). Exchange simulations indicate that the evaluation parameter of the total exchange divided by the Gini index can be expressed by the same saturated curvilinear approximate equation using the wealth transfer rate and time period of redistribution and the surplus contribution rate of the wealthy and the saving rate. However, considering the coercion of taxes and its associated costs and independence based on the morality of mutual aid, a nonequivalent exchange without return obligation is preferred. This is oriented toward Graeber's baseline communism and Karatani's mode of exchange D, with implications for alternatives to the capitalist economy.