I suggest finding a better example to motivate this approach. Few minor corrections: 1) Some of the citations are referred as arxiv submissions but are journal publications, so please update references (e.g [3], [10], [18], [24] -- which is also missing author information.

All 4 reviewers suggest that this paper is above the acceptance threshold. After reading the reviews and author response, I will recommend acceptance with a note (see bellow). Reviewers agree that the idea of filtering what information from other agents to consider within a multi-agent setup, while not new, is an important area of research, especially as the number of agents and the complexity of the environments grows. All reviewers agree that this is a sound submission, with good experimental results and comprehensive comparisons with previous work. Some concerns were raised, but the author response did a good job in addressing many of those.

Weaknesses: 1) The task of enhancing the target coverage in Directional Sensor Networks (DSNs) is important and challenging. However, as far as I am concerned, it is not a standard benchmark environment for studying multi-agent reinforcement learning. The proposed method/model design targets at a specific problem, limiting its significance. There already exist some popular environments for multi-agent cooperation. If experiments are conducted on these standard benchmarks, the significance of this work for the machine learning (ML) or reinforcement learning (RL) community can be improved.

This paper proposes multi-agent hierarchical RL method to the target coverage problems in directional sensor networks. Empirical results are provided to show the advantage of their method against state of the art MARL algorithms as well as optimization techniques specific to the target coverage problem. There are some concerns among the reviewers regarding whether RL is the right tool for the problem, insufficient comparison with non-learning heuristics, and the value of the work to the RL community. I share the first reviewer's positive sentiment on the application of RL to sensor networks. It is nice to see RL moving from games to real-world applications.

The paper builds on well-known methods (CFR) and provides novel improvements and modifications that extend the approach to a multiplayer, hidden-role setting. This is original and novel and creative, though the crucial role of CFR cannot be understated. Related work appears to be adequately cited. The empirical results provide the main validation for the soundness and quality of the proposed algorithm; this is reasonable and is explained well in the paper. I have not spotted any obvious illogicalities or mistakes.

All reviewers agree that the paper provides some nice contributions (extending CFR beyond 2 players and tackling Avalon) and that the authors succeed well with their rebuttal to address some of the major concerns brought on by some of the referees. They have responded adequately and furthermore open-sourced their implementation. We expect the authors though to carry out the promised changes (and also improve on the notation).

This paper tackles the problem of decentralized learning in multi-agent environments. While many recent approaches use a combination of centralized learning and decentralized execution, the decentralized learning paradigm is motivated by scenarios where a centralized agent (e.g. a value function) may be too expensive to use, or may have undesirable privacy implications. However, previous decentralized learning approaches haven't been very effective for multi-agent problems. The paper proposes a new algorithm, value propagation, and prove that it converges in the non-linear function approximation case. To my knowledge, the value propagation algorithm is novel and interesting.

Originality: Q: Are the tasks or methods new? A: The task is not new, the method is new. Q: Is the work a novel combination of well-known techniques? A: The work is a novel combination of well-known techniques. Q: Is it clear how this work differs from previous contributions?

The dynamically changing alliances mean that the domain of diplomacy presents unique challenges for agents. I agree with the authors that this means that diplomacy is'deserving of special attention', I would consider the full game to be a grand challenge for multi-agent research. With recent progress in large-scale RL focusing on single-agent and 2-player zero sum games, this problem is particularly timely. This work presents state of the art agents trained with deep learning. To my knowledge this is the first successful application of deep learning to diplomacy.

All reviewers agree that this paper explores interesting territory, i.e., multi-agent Learning in the Diplomacy game. It is a well written and presented paper. The paper has generated quite some discussion after the rebuttal, discussing all pros and cons of the work. The major point in favor of the work (as also indicated by the authors themselves) seems to be that the work lays some ground work for future research in the Diplomacy game, that is known to be very hard and challenging. The biggest point of concern is that the paper presents little innovation in the techniques that it deploys but rather shows how the SOTA can be used/engineered to be successful in this domain to a certain extent, and illustrates the performance of known algorithms.