To handle this problem, a natural approach is to increase the data diversity by image based augmentations. However, different with most vision tasks such as classification and detection, RL tasks are not always invariant to spatial based augmentations duetotheentanglement ofenvironment dynamics andvisual appearance.

Deep reinforcement learning (RL) is computationally demanding and requiresprocessing of many data points. Synchronous methods enjoy training stability while having lowerdatathroughput.

The paper was reviewed by experts on the topic and discussed after authors rebuttal. Results were found to be interesting and valuable. The reviewers comments should be taken into account while preparing the final version of the paper.