This supplementary document is structured as follows.

Active perception,collecting observations to reduce uncertainty about ahidden variable, isone of the fundamental capabilities of an intelligent agent [2]. In multi-agent active perceptiona team of autonomous agents cooperatively gathers observations to infer the value of a hidden variable.

Agent 1's action is resolved first. Figure 8: An example of Agent 1 using the "clean" action while facing East. The "main" beam extends directly in front of the agent, while two auxiliary A beam stops when it hits a dirty river tile. The Sequential Social Dilemma Games, introduced in Leibo et al. [2017], are a kind of MARL All of these have open source implementations in [Vinitsky et al., 2019]. The cleaning beam is shown in Figure 8a.

This paper introduces the PettingZoo library and the accompanying Agent Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets of multi-agent environments with a universal, elegant Python API. PettingZoo was developed with the goal of accelerating research in Multi-Agent Reinforcement Learning ("MARL "), by making work more interchangeable, accessible and reproducible akin to what OpenAI's Gym library did for single-agent reinforcement

To enable decentralized execution, we alsoderivefactorized per-agentpolicies inspired byamaximum-entropyMARL framework.