Reviews: Distral: Robust multitask reinforcement learning

The paper presents an approach to performing transfer between multiple reinforcement learning tasks by regularizing the policies of different tasks towards a central policy, and also encouraging exploration in these policies. The approach relies on KL-divergence regularization. The idea is straightforward and well explained. There are no theoretical results regarding the learning speed or quality of the policies obtained (though these are soft, so clearly there would be some performance loss compared to optimal). The evaluation shows slightly better results that A3C baselines in both some simple mazes and deep net learning tasks. While the paper is well written, and the results are generally positive, the performance improvements are modest.