In many multi-agent settings, participants can form teams to achieve collective outcomes that may far surpass their individual capabilities.

Several works haveshown this vulnerability via adversarial attacks, butexisting approaches onimproving therobustness ofDRL under this setting have limited success and lack for theoretical principles. We show that naively applying existing techniques on improving robustness for classification tasks,likeadversarialtraining,areineffectiveformanyRLtasks.

Theactiveagent trains (as a regular Double-DQN) up to the time of forking, at which point the passive agent is created asa'fork' (i.e.,with identical networkweights) oftheactiveagent.

Inthiswork, we carefully study the performance of PPO in cooperative multi-agent settings.

Reinforcement Learning agent and a configurator that can modify some environmental parameters to improve the agent's performance.

However, in adversarial environments, these methods can be non-robust, since the return is dependent on the strategies of both the decision-maker and adversary. Training a probabilistic model conditioned on observed return to predict action can fail to generalize, as the trajectories that achieve a return in the dataset might have done so due to a suboptimal behavior adversary.