In Section 4.2, we have shown the effectiveness of In Section 3.4, we have analyzed that I2Q can easily solve the task with multiple optimal joint policies. Here, we give another way to solve this problem. D3G cannot obtain a winning rate in SMAC, as shown in Table 1. Although QSS value is a biased estimation in this implementation, the implementation without forward model is practical. The results are shown in Figure 16.

Reinforcement learning (RL) algorithms can find an optimal policy for a single agent to accomplish a particular task. However, many real-world problems require multiple agents to collaborate in order to achieve a common goal. For example, a robot executing a task in a warehouse may require the assistance of a drone to retrieve items from high shelves. In Decentralized Multi-Agent RL (DMARL), agents learn independently and then combine their policies at execution time, but often must satisfy constraints on compatibility of local policies to ensure that they can achieve the global task when combined. In this paper, we study how providing high-level symbolic knowledge to agents can help address unique challenges of this setting, such as privacy constraints, communication limitations, and performance concerns. In particular, we extend the formal tools used to check the compatibility of local policies with the team task, making decentralized training with theoretical guarantees usable in more scenarios. Furthermore, we empirically demonstrate that symbolic knowledge about the temporal evolution of events in the environment can significantly expedite the learning process in DMARL.

In Section 4.2, we have shown the effectiveness of In Section 3.4, we have analyzed that I2Q can easily solve the task with multiple optimal joint policies. Here, we give another way to solve this problem. D3G cannot obtain a winning rate in SMAC, as shown in Table 1. Although QSS value is a biased estimation in this implementation, the implementation without forward model is practical. The results are shown in Figure 16.

The paper presents a new exploration strategy for decentralized MARL that is based on a joint latent variable that is shared between the agent. This paper is a difficult case. While the theoretical insights concerning the difficulty of the exploration problem in decentralized MARL are insightful, the experimental results were not good enough in the original submission to convince the reviewers. The algorithm was only in one case considerably better than the competitor QMix and other baseline comparison were missing. However, in the rebuttal the authors provided much better results as well as additional comparison to Qtrans.

Weaknesses: - The biggest weakness of this paper in my mind is the clarity and framing. The paper motivates the contribution by stating that agents may not have access to the reward functions / models of other agents. For example, the paper states: "In many practical applications, the agents may not have perfect information of the model, i.e., the reward function and/or the transition probability model. For example, in an urban traffic network that involves multiple self-driving cars, each vehicle makes an individual action and has no access to other cars' rewards and models." However, most MARL methods don't make any assumptions about the reward function of other agents, particularly in the decentralized MARL setting.