Thispaperproposes animplicit model-based multi-agent reinforcement learning method based onvalue decomposition methods. Under this method, agents can interact with thelearned virtual environment and evaluate thecurrent state value according to imagined future states in the latent space, making agents have the foresight. Our approach can be applied toanymulti-agent value decomposition method.

Recently,anincreasing number ofalgorithms for curriculum generation havebeen proposed, empirically demonstrating that CL is an appropriate tool to improve the sample efficiency of DRL algorithms [9, 10]. However, these algorithms are based on heuristics and concepts that are, as ofnow,theoretically notwell understood, preventing theestablishment ofrigorous improvements. In contrast, we propose to generate the curriculum based on a principled inference view on RL. Our approach generates the curriculum based on two quantities: The value function of the agent and the KL divergence to a target distribution of tasks.

Moreover, we extend ICQ to multi-agent tasks by decomposing the joint-policy under the implicit constraint. Experimental results demonstrate that the extrapolation error is successfully controlled within a reasonable range and insensitive to the number of agents.