In multiagent domains, coping with non-stationary agents that change behaviors from time to time is a challenging problem, where an agent is usually required to be able to quickly detect the other agent's policy during online interaction, and then adapt its own policy accordingly.

The paper focuses on an important problem in multiagent learning - non-stationarity introduced by other agents. It proposes a novel rectified belief model to overcome the problem of indistinguishability with miscoordinated policies and combines a few ideas made popular by neural networks - sharing weights and distillation. This results in an extension of the idea of Bayesian Policy reuse, originally formulated for transfer learning and later extended into BPR for online learning, which the paper terms Deep BPR . The paper tests the efficacy of their approach on relatively small tasks and finds that the proposed method can perform quite close to an omniscient one. The paper clearly traces the origin of its ideas to BPR and BPR algorithms and the limitations it's trying to overcome.

In multiagent domains, coping with non-stationary agents that change behaviors from time to time is a challenging problem, where an agent is usually required to be able to quickly detect the other agent's policy during online interaction, and then adapt its own policy accordingly. We propose a new deep BPR algorithm by extending the recent BPR algorithm with a neural network as the value-function approximator. To detect policy accurately, we propose the \textit{rectified belief model} taking advantage of the \textit{opponent model} to infer the other agent's policy from reward signals and its behaviors. Instead of directly storing individual policies as BPR, we introduce \textit{distilled policy network} that serves as the policy library in BPR, using policy distillation to achieve efficient online policy learning and reuse. Deep BPR inherits all the advantages of BPR and empirically shows better performance in terms of detection accuracy, cumulative rewards and speed of convergence compared to existing algorithms in complex Markov games with raw visual inputs.