Learning in multi-agent systems is highly challenging due to the inherent complexity introduced by agents' interactions. We tackle systems with a huge population of interacting agents (e.g., swarms) via Mean-Field Control (MFC). MFC considers an asymptotically infinite population of identical agents that aim to collaboratively maximize the collective reward. Specifically, we consider the case of unknown system dynamics where the goal is to simultaneously optimize for the rewards and learn from experience. We propose an efficient model-based reinforcement learning algorithm $\text{M}^3\text{-UCRL}$ that runs in episodes and provably solves this problem. $\text{M}^3\text{-UCRL}$ uses upper-confidence bounds to balance exploration and exploitation during policy learning. Our main theoretical contributions are the first general regret bounds for model-based RL for MFC, obtained via a novel mean-field type analysis. $\text{M}^3\text{-UCRL}$ can be instantiated with different models such as neural networks or Gaussian Processes, and effectively combined with neural network policy learning. We empirically demonstrate the convergence of $\text{M}^3\text{-UCRL}$ on the swarm motion problem of controlling an infinite population of agents seeking to maximize location-dependent reward and avoid congested areas.

Falsification has emerged as an important tool for simulation-based verification of autonomous systems. In this paper, we present extensions to the Scenic scenario specification language and VerifAI toolkit that improve the scalability of sampling-based falsification methods by using parallelism and extend falsification to multi-objective specifications. We first present a parallelized framework that is interfaced with both the simulation and sampling capabilities of Scenic and the falsification capabilities of VerifAI, reducing the execution time bottleneck inherently present in simulation-based testing. We then present an extension of VerifAI's falsification algorithms to support multi-objective optimization during sampling, using the concept of rulebooks to specify a preference ordering over multiple metrics that can be used to guide the counterexample search process. Lastly, we evaluate the benefits of these extensions with a comprehensive set of benchmarks written in the Scenic language.

Text Worlds are virtual environments for embodied agents that, unlike 2D or 3D environments, are rendered exclusively using textual descriptions. These environments offer an alternative to higher-fidelity 3D environments due to their low barrier to entry, providing the ability to study semantics, compositional inference, and other high-level tasks with rich high-level action spaces while controlling for perceptual input. This systematic survey outlines recent developments in tooling, environments, and agent modeling for Text Worlds, while examining recent trends in knowledge graphs, common sense reasoning, transfer learning of Text World performance to higher-fidelity environments, as well as near-term development targets that, once achieved, make Text Worlds an attractive general research paradigm for natural language processing.

In the following sections, we briefly review the necessary material from federated learning and mean-field games that will be used later when similarities start to appear. In federated learning (Section 1.1), we present the general idea and one of its standard algorithm called Federated Averaging (FedAvg). The general setting of stochastic games is introduced in Section 2. Two main pillars of mean-field games that are stochastic optimal control and stochastic differential games are introduced in Sections 2.1 and 2.2 respectively. The introduction of mean-field games is completed in Section 2.3. Finally, the connection between federated learning and mean-field games is established in Section 3.

Imitation learning enables agents to reuse and adapt the hard-won expertise of others, offering a solution to several key challenges in learning behavior. Although it is easy to observe behavior in the real-world, the underlying actions may not be accessible. We present a new method for imitation solely from observations that achieves comparable performance to experts on challenging continuous control tasks while also exhibiting robustness in the presence of observations unrelated to the task. Our method, which we call FORM (for "Future Observation Reward Model") is derived from an inverse RL objective and imitates using a model of expert behavior learned by generative modelling of the expert's observations, without needing ground truth actions. We show that FORM performs comparably to a strong baseline IRL method (GAIL) on the DeepMind Control Suite benchmark, while outperforming GAIL in the presence of task-irrelevant features.

Agent Based Modelling (ABM) is a computational framework for simulating the behaviours and interactions of autonomous agents. As Agent Based Models are usually representative of complex systems, obtaining a likelihood function of the model parameters is nearly always intractable. There is a necessity to conduct inference in a likelihood free context in order to understand the model output. Approximate Bayesian Computation is a suitable approach for this inference. It can be applied to an Agent Based Model to both validate the simulation and infer a set of parameters to describe the model. Recent research in ABC has yielded increasingly efficient algorithms for calculating the approximate likelihood. These are investigated and compared using a pedestrian model in the Hamilton CBD.

Let's start from the broader classification. Distributed Artificial Intelligence (DAI) is a class of technologies and methods that span from swarm intelligence to multi-agent technologies and that basically concerns the development of distributed solutions for a specific problem. It can mainly be used for learning, reasoning, and planning, and it is one of the subsets of AI where simulation has a way greater importance than point-prediction. In this class of systems, autonomous learning processing agents (distributed at large scale and independent) reach conclusions or a semi-equilibrium through interaction and communication (even asynchronously). One of the big benefits of those with respect to neural networks is that they do not require the same amount of data to work -- far to say though these are simple systems.

Two-sided matching markets, such as Uber, Airbnb, stock markets, and dating apps, play a significant role in today's world. As a result, there is a tremendous and rising interest to design better mechanisms for two-sided matching. The seminal work of Gale and Shapley [14] introduced a simple mechanism for stable matching in two-sided markets--Deferred-acceptance (DA)--which has since has been applied in doctor-hospital matching [25], school choice [3, 22, 2], and the matching of cadets to their branches of military service [30, 29]. DA is stable, i.e., no pair of agents mutually prefer each other to their DA partners. On the other hand, DA is not strategy-proof (SP); that is, under fully general preferences, it is always possible that some agent can mis-report her preferences to obtain a better matching than she would receive under the DA mechanism.

Adaptive traffic signal control (ATSC) in urban traffic networks poses a challenging task due to the complicated dynamics arising in traffic systems. In recent years, several approaches based on multi-agent deep reinforcement learning (MARL) have been studied experimentally. These approaches propose distributed techniques in which each signalized intersection is seen as an agent in a stochastic game whose purpose is to optimize the flow of vehicles in its vicinity. In this setting, the systems evolves towards an equilibrium among the agents that shows beneficial for the whole traffic network. A recently developed multi-agent variant of the well-established advantage actor-critic (A2C) algorithm, called MA2C (multi-agent A2C) exploits the promising idea of some communication among the agents. In this view,the agents share their strategies with other neighbor agents, thereby stabilizing the learning process even when the agents grow in number and variety. We experimented MA2C in two traffic networks located in Bologna (Italy) and found that its action translates into a significant decrease of the amount of pollutants released into the environment.

In shared spaces, motorized and non-motorized road users share the same space with equal priority. Their movements are not regulated by traffic rules, hence they interact more frequently to negotiate priority over the shared space. To estimate the safeness and efficiency of shared spaces, reproducing the traffic behavior in such traffic places is important. In this paper, we consider and combine different levels of interaction between pedestrians and cars in shared space environments. Our proposed model consists of three layers: a layer to plan trajectories of road users; a force-based modeling layer to reproduce free flow movement and simple interactions; and a game-theoretic decision layer to handle complex situations where road users need to make a decision over different alternatives. We validate our model by simulating scenarios involving various interactions between pedestrians and cars and also car-to-car interaction. The results indicate that simulated behaviors match observed behaviors well.