On the Convergence of Smooth Regularized Approximate Value Iteration Schemes

A number of techniques is commonly used in the large-scale RL setting, namely, entropy regularization, smoothing of Q-values and neural network function approximation.