Existing methods for agent modelling commonly assume knowledge of the local observations and chosen actions of the modelled agents during execution. To eliminate this assumption, we extract representations from thelocalinformation ofthecontrolled agent using encoderdecoderarchitectures.

The modeling of ideal transition function inI2Q isfully decentralized and independent from the learned policies of other agents, helping I2Q be free from non-stationarity and learn the optimal policy.

To handle this problem, a natural approach is to increase the data diversity by image based augmentations. However, different with most vision tasks such as classification and detection, RL tasks are not always invariant to spatial based augmentations duetotheentanglement ofenvironment dynamics andvisual appearance.

Deep reinforcement learning (RL) is computationally demanding and requiresprocessing of many data points. Synchronous methods enjoy training stability while having lowerdatathroughput.