Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning

Left: The reward function is a mixture of Gaussian, and the offline data distribution is unbalanced with most samples located in low-reward states.