Reviews: Biases for Emergent Communication in Multi-agent Reinforcement Learning

The authors present two losses for improving emergent communication, addressing concerns laid out in previous work (Lowe 2019). One is the concern that the speaker agent may be communicating generic messages and not ones relevant to the particular sitation. A loss here encourages the agent to send messages that are correlated with their observation, based on maximizing mutual information between them. A second concern is that the listener agent may not be conditioning their behavior on the communication, and in this case an extra loss Both constraints are intuitive, and phrasing them as losses doesn't seem to be particularly challenging. However, as with many issues in emergent communication, such a judgement may gloss over hidden difficulties in a complex optimization problem, and this appears to be the case here, requiring some non-obvious sidestepping to provide losses with better convergence.