Variational Deep Q Network

We propose a framework that directly tackles the probability distribution of the value function parameters in Deep Q Network (DQN), with powerful variational inference subroutines to approximate the posterior of the parameters. We will establish the equivalence between our proposed surrogate objective and variational inference loss. Our new algorithm achieves efficient exploration and performs well on large scale chain Markov Decision Process (MDP). Deep reinforcement learning (RL) has enjoyed numerous recent successes in video games, board games, and robotics control [17, 3, 9, 18]. Deep RL algorithms typically apply naive exploration schemes such as ɛ greedy [12, 19], directly injecting noise into actions [10], and action level entropy regularization [24].