Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal. Reliance on trust and coordination makes Diplomacy the first non-cooperative multi-agent benchmark for complex sequential social dilemmas in a rich environment. In this work, we focus on training an agent that learns to play the No Press version of Diplomacy where there is no dedicated communication channel between players.

The dynamically changing alliances mean that the domain of diplomacy presents unique challenges for agents. I agree with the authors that this means that diplomacy is'deserving of special attention', I would consider the full game to be a grand challenge for multi-agent research. With recent progress in large-scale RL focusing on single-agent and 2-player zero sum games, this problem is particularly timely. This work presents state of the art agents trained with deep learning. To my knowledge this is the first successful application of deep learning to diplomacy.

All reviewers agree that this paper explores interesting territory, i.e., multi-agent Learning in the Diplomacy game. It is a well written and presented paper. The paper has generated quite some discussion after the rebuttal, discussing all pros and cons of the work. The major point in favor of the work (as also indicated by the authors themselves) seems to be that the work lays some ground work for future research in the Diplomacy game, that is known to be very hard and challenging. The biggest point of concern is that the paper presents little innovation in the techniques that it deploys but rather shows how the SOTA can be used/engineered to be successful in this domain to a certain extent, and illustrates the performance of known algorithms.

Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal. Reliance on trust and coordination makes Diplomacy the first non-cooperative multi-agent benchmark for complex sequential social dilemmas in a rich environment. In this work, we focus on training an agent that learns to play the No Press version of Diplomacy where there is no dedicated communication channel between players. The model was trained on a new dataset of more than 150,000 human games. Our model is trained by supervised learning (SL) from expert trajectories, which is then used to initialize a reinforcement learning (RL) agent trained through self-play.

Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal. Reliance on trust and coordination makes Diplomacy the first non-cooperative multi-agent benchmark for complex sequential social dilemmas in a rich environment. In this work, we focus on training an agent that learns to play the No Press version of Diplomacy where there is no dedicated communication channel between players. The model was trained on a new dataset of more than 150,000 human games. Our model is trained by supervised learning (SL) from expert trajectories, which is then used to initialize a reinforcement learning (RL) agent trained through self-play.