Appendix Gigastep - One Billion Steps per Second Multi-agent Reinforcement Learning

In competitive or adversarial MARL, an objective reward measure is not defined, as the collected reward inherently depends on the relative strength of the opposing agent's policy. For instance, for the identical 5 vs 5 scenario, in Figure 1a, we plot the win rate of a policy at different checkpoints (x-axis) compared to the same policy at every other checkpoint of the training process (y-axis). For instance, in Figure 7, we show the win rates from the perspective of the team A policy. In this section, we describe the behaviors that emerged after the training process of some of Gigastep's This analysis serves as a demonstration that the task considered in Gigastep allows intelligent and collaborative behavior to emerge through MARL algorithms. We study the identical 20 vs 20 and the special 5 vs 1 scenarios here. A diverse set of behaviors was discovered using the baseline training method.