Our initial experiments (implementation, debugging, hyperparameter tuning, etc.) required about 5000CPUhoursofcompute. Due to these rules, it is recommended to group together in order to attack simultaneously. In Warehouse[4], QTRAN makes slightly faster progress than VAST(η = 12). The results forWarehouse[16], Battle[80], and GaussianSqueeze[800] are shown in Figure 1. Figure 10: Visualizations of the generated sub-teams ofXMetaGrad with η = 14 and XSpatial with k-means clustering using 10 centroids at different stages (early, middle, late) inBattle[80] after training. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments.

Value function factorization (VFF) is a popular approach to cooperative multiagent reinforcement learning in order to learn local value functions from global rewards.

When we state "all agents are required to take decreasing step61 sizes", wewere particularly referring toresults presented in[9,33,41].

In many multi-agent settings, participants can form teams to achieve collective outcomes that may far surpass their individual capabilities.

Several works haveshown this vulnerability via adversarial attacks, butexisting approaches onimproving therobustness ofDRL under this setting have limited success and lack for theoretical principles. We show that naively applying existing techniques on improving robustness for classification tasks,likeadversarialtraining,areineffectiveformanyRLtasks.

Theactiveagent trains (as a regular Double-DQN) up to the time of forking, at which point the passive agent is created asa'fork' (i.e.,with identical networkweights) oftheactiveagent.