Opinions on implementing dropout in RNN/LSTMs? • /r/MachineLearning

I'm getting some practice with RNN and LSTMs right now, and honestly I'm a little afraid of overfitting; It just feels a lot harder to avoid with these models over CNNs or vanilla NNs. For now I'm working with one hidden layer since I don't expect my data to be bigger than perhaps 1.5 MBs of characters. I'm just looking for some opinions on where dropout should be implemented Please correct me if I'm approaching dropout improperly with RNN/LSTMs. My preliminary basic idea was to add dropout on the hidden states (maybe @ 0.5 to safely begin with? I can cross validate the HP likely), leaving the LSTM cell states as is.