[P] Trying out DQN with gym and keras. Bug? • r/MachineLearning

Do I have a bug, is it a matter of tweaking hyperparameters, or is the algorithm simply not that suited for these problems? The best performing submissions for cartpole you see on openai gym are discrete q learning with a q table represented as a lookup table. Because cartpole is simple you can discretize the 1x4 real valued state vector into a 1x4 integer vector with 8 or so bins per channel and then store a 84 element q table explicitly in memory. A neural net, even a small one is overkill as an approximation of this fixed sized q table so naturally the former performs better. Hyper parameters are a bitch to tune, and q learning isn't remotely stable 99% of the time until you balance them well enough so it can easily seem like nothing is working.