Exploration in multi-agent reinforcement learning is a challenging problem, especially in environments with sparse rewards. We propose a general method for efficient exploration by sharing experience amongst agents.

Neural Information Processing Systems http://nips.cc/

TD converges at a sublinear rate to the global optimum of the mean-squared projected Bellman error for policy evaluation.

The resultant formulation can then be tackled using a neural GTD algorithm.

The resultant formulation can then be tackled using a neural GTD algorithm.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The paper proposes and analyzes a method for learning in robust MDPs. While this setting is very similar to learning in stochastic games, the main difference is that in robust games, the optimal move of the opponent is observed, while in robust MDPs the decision maker only observes the outcome (the opponent chooses the probabilities). The paper makes a small advance on a relevant, non-trivial, and interesting topic, but I am not sure that it is quite ready for publication in its current form. First, the setting is somewhat contrived and not motivated. A natural setting would be simply to use reinforcement learning to learn to act in a robust setting.

Neural Information Processing Systems http://nips.cc/