Inspired by cognitive theories of creativity, this paper introduces a computational model (AIGenC) that lays down the necessary components to enable artificial agents to learn, use and generate transferable representations. Unlike machine representation learning, which relies exclusively on raw sensory data, biological representations incorporate relational and associative information that embeds rich and structured concept spaces. The AIGenC model poses a hierarchical graph architecture with various levels and types of representations procured by different components. The first component, Concept Processing, extracts objects and affordances from sensory input and encodes them into a concept space. The resulting representations are stored in a dual memory system and enriched with goal-directed and temporal information acquired through reinforcement learning, creating a higher-level of abstraction. Two additional components work in parallel to detect and recover relevant concepts and create new ones, respectively, in a process akin to cognitive Reflective Reasoning and Blending. The Reflective Reasoning unit detects and recovers from memory concepts relevant to the task by means of a matching process that calculates a similarity value between the current state and memory graph structures. Once the matching interaction ends, rewards and temporal information are added to the graph, building further abstractions. If the reflective reasoning processing fails to offer a suitable solution, a blending operation comes into place, creating new concepts by combining past information. We discuss the model's capability to yield better out-of-distribution generalisation in artificial agents, thus advancing toward Artificial General Intelligence.

This paper presents a real-time trajectory planning framework for Urban Air Mobility (UAM) that is both safe and scalable. The proposed framework employs a decentralized, free-flight concept of operation in which each aircraft independently performs separation assurance and conflict resolution, generating safe trajectories by accounting for the future states of nearby aircraft. The framework consists of two main components: a data-driven reachability analysis tool and an efficient Markov Decision Process (MDP) based decision maker. The reachability analysis over-approximates the reachable set of each aircraft through a discrepancy function learned online from simulated trajectories. The decision maker, on the other hand, uses a 6-degrees-of-freedom guidance model of fixed-wing aircraft to ensure collision-free trajectory planning. Additionally, the proposed framework incorporates reward shaping and action shielding techniques to enhance safety performance. The proposed framework is evaluated through simulation experiments involving up to 32 aircraft in a UAM setting, with performance measured by the number of Near Mid Air Collisions (NMAC) and computational time. The results demonstrate the safety and scalability of the proposed framework.

Network slicing enables operators to efficiently support diverse applications on a common physical infrastructure. The ever-increasing densification of network deployment leads to complex and non-trivial inter-cell interference, which requires more than inaccurate analytic models to dynamically optimize resource management for network slices. In this paper, we develop a DIRP algorithm with multiple deep reinforcement learning (DRL) agents to cooperatively optimize resource partition in individual cells to fulfill the requirements of each slice, based on two alternative reward functions. Nevertheless, existing DRL approaches usually tie the pretrained model parameters to specific network environments with poor transferability, which raises practical deployment concerns in large-scale mobile networks. Hence, we design a novel transfer learning-aided DIRP (TL-DIRP) algorithm to ease the transfer of DIRP agents across different network environments in terms of sample efficiency, model reproducibility, and algorithm scalability. The TL-DIRP algorithm first centrally trains a generalized model and then transfers the "generalist" to each local agent as "specialist" with distributed finetuning and execution. TL-DIRP consists of two steps: 1) centralized training of a generalized distributed model, 2) transferring the "generalist" to each "specialist" with distributed finetuning and execution. The numerical results show that not only DIRP outperforms existing baseline approaches in terms of faster convergence and higher reward, but more importantly, TL-DIRP significantly improves the service performance, with reduced exploration cost, accelerated convergence rate, and enhanced model reproducibility. As compared to a traffic-aware baseline, TL-DIRP provides about 15% less violation ratio of the quality of service (QoS) for the worst slice service and 8.8% less violation on the average service QoS.

The evolution of cooperation has been investigated intensely in various disciplines, such as biology, economics, computer science, physics and psychology. There are two important dimensions, among many (Bowles and Gintis, 2011; Lehmann and Keller, 2006; Nowak, 2006), that have been shown to affect the evolution of cooperation: the interaction structure, i.e., who interacts with whom (Santos et al., 2006), and the mode of cognition, i.e., the extent of deliberation as opposed to intuition (Capraro, 2019). While for the interaction structure there is a substantial consensus that sparse and heavily clustered networks help the spread of cooperation (Nowak, 2006; Ohtsuki et al., 2006), for the mode of cognition results are more articulated and depend on specific features of the social dilemma (Bear et al., 2017; Bear and Rand, 2016) and of the cost of deliberation (Jagau and van Veelen, 2017). An important aspect in evolutionary models is the behavioral rule adopted by agents, which heavily contributes to determining the trajectories of the dynamic adjustment. While the literature has extensively considered behavioral rules encompassing best reply (Bilancini and Boncinelli, 2009) and imitation (Levine and Pesendorfer, 2007) as well as processes of the type death-birth or birth-death (Ohtsuki et al., 2006), little attention has been given to evolutionary dynamics based on reinforcement learning (Tanabe and Masuda, 2012). Reinforcement learning is a prominent behavioral rule originated in behavioral sciences (Skinner, 1938a,b) and recently become extremely popular in computer sciences, with many different applications (Nian et al., 2020).

Cooperative multi-agent reinforcement learning (MARL) for navigation enables agents to cooperate to achieve their navigation goals. Using emergent communication, agents learn a communication protocol to coordinate and share information that is needed to achieve their navigation tasks. In emergent communication, symbols with no pre-specified usage rules are exchanged, in which the meaning and syntax emerge through training. Learning a navigation policy along with a communication protocol in a MARL environment is highly complex due to the huge state space to be explored. To cope with this complexity, this work proposes a novel neural network architecture, for jointly learning an adaptive state space abstraction and a communication protocol among agents participating in navigation tasks. The goal is to come up with an adaptive abstractor that significantly reduces the size of the state space to be explored, without degradation in the policy performance. Simulation results show that the proposed method reaches a better policy, in terms of achievable rewards, resulting in fewer training iterations compared to the case where raw states or fixed state abstraction are used. Moreover, it is shown that a communication protocol emerges during training which enables the agents to learn better policies within fewer training iterations.

Abstract-- Motion prediction for automated vehicles in complex environments is a difficult task that is to be mastered when automated vehicles are to be used in arbitrary situations. Many factors influence the future motion of traffic participants starting with traffic rules and reaching from the interaction between each other to personal habits of human drivers. Therefore, we present a novel approach for a graph-based prediction based on a heterogeneous holistic graph representation that combines temporal information, properties and relations between traffic participants as well as relations with static elements such as the road network. The information is encoded through different types of nodes and edges that both are enriched with arbitrary features. We evaluated the approach on the INTERACTION and the Argoverse dataset and conducted an informative ablation study to demonstrate the benefit of different types of information for the motion prediction quality. Map-nodes are depicted in yellow.

In multi-agent reinforcement learning, the use of a global objective is a powerful tool for incentivising cooperation. Unfortunately, it is not sample-efficient to train individual agents with a global reward, because it does not necessarily correlate with an agent's individual actions. This problem can be solved by factorising the global value function into local value functions. Early work in this domain performed factorisation by conditioning local value functions purely on local information. Recently, it has been shown that providing both local information and an encoding of the global state can promote cooperative behaviour. In this paper we propose QGNN, the first value factorisation method to use a graph neural network (GNN) based model. The multi-layer message passing architecture of QGNN provides more representational complexity than models in prior work, allowing it to produce a more effective factorisation. QGNN also introduces a permutation invariant mixer which is able to match the performance of other methods, even with significantly fewer parameters. We evaluate our method against several baselines, including QMIX-Att, GraphMIX, QMIX, VDN, and hybrid architectures. Our experiments include Starcraft, the standard benchmark for credit assignment; Estimate Game, a custom environment that explicitly models inter-agent dependencies; and Coalition Structure Generation, a foundational problem with real-world applications. The results show that QGNN outperforms state-of-the-art value factorisation baselines consistently.

In the classic network security games, the defender distributes defending resources to the nodes of the network, and the attacker attacks a node, with the objective of maximizing the damage caused. In this paper, we consider the network defending problem against contagious attacks, e.g., the attack at a node u spreads to the neighbors of u and can cause damage at multiple nodes. Existing works that study shared resources assume that the resource allocated to a node can be shared or duplicated between neighboring nodes. However, in the real world, sharing resource naturally leads to a decrease in defending power of the source node, especially when defending against contagious attacks. Therefore, we study the model in which resources allocated to a node can only be transferred to its neighboring nodes, which we refer to as a reallocation process. We show that the problem of computing optimal defending strategy is NP -hard even for some very special cases. For positive results, we give a mixed integer linear program formulation for the problem and a bi-criteria approximation algorithm. Our experimental results demonstrate that the allocation and reallocation strategies our algorithm computes perform well in terms of minimizing the damage due to contagious attacks.

Achieving the economical and stable operation of Multi-microgrids (MMG) systems is vital. However, there are still some challenging problems to be solved. Firstly, from the perspective of stable operation, it is necessary to minimize the energy fluctuation of the main grid. Secondly, the characteristics of energy conversion equipment need to be considered. Finally, privacy protection while reducing the operating cost of an MMG system is crucial. To address these challenges, a Data-driven strategy for MMG systems with Shared Energy Storage (SES) is proposed. The Mixed-Attention is applied to fit the conditions of the equipment, additionally, Multi-Agent Soft Actor-Critic(MA-SAC) and (Multi-Agent Win or Learn Fast Policy Hill-Climbing)MA-WoLF-PHC are proposed to solve the partially observable dynamic stochastic game problem. By testing the operation data of the MMG system in Northwest China, following conclusions are drawn: the R-Square (R2) values of results reach 0.999, indicating the neural network effectively models the nonlinear conditions. The proposed MMG system framework can reduce energy fluctuations in the main grid by 1746.5kW in 24 hours and achieve a cost reduction of 16.21% in the test. Finally, the superiority of the proposed algorithms is verified through their fast convergence speed and excellent optimization performance.

Trajectory prediction has been a long-standing problem in intelligent systems like autonomous driving and robot navigation. Models trained on large-scale benchmarks have made significant progress in improving prediction accuracy. However, the importance on efficiency for real-time applications has been less emphasized. This paper proposes an attention-based graph model, named GATraj, which achieves a good balance of prediction accuracy and inference speed. We use attention mechanisms to model the spatial-temporal dynamics of agents, such as pedestrians or vehicles, and a graph convolutional network to model their interactions. Additionally, a Laplacian mixture decoder is implemented to mitigate mode collapse and generate diverse multimodal predictions for each agent. GATraj achieves state-of-the-art prediction performance at a much higher speed when tested on the ETH/UCY datasets for pedestrian trajectories, and good performance at about 100 Hz inference speed when tested on the nuScenes dataset for autonomous driving. We conduct extensive experiments to analyze the probability estimation of the Laplacian mixture decoder and compare it with a Gaussian mixture decoder for predicting different multimodalities. Furthermore, comprehensive ablation studies demonstrate the effectiveness of each proposed module in GATraj. The code is released at https://github.com/mengmengliu1998/GATraj.