I am not sure did I understand it correctly. In each state, we update the score of chosen action to be the best Q-Value of the next state and keep the score of other action to be unchanged. Moreover, we put state, updated scores of all actions into memory. We sample N pairs in the memory (needed to be in the same game??) and train them altogether. So we only calculate the new score of the transition that we just take and reuse the calculated scores of previous transition stored in memory?

There are also many good books that focus on one particular topic. For example, Sutton and Barto's Reinforcement Learning is a classic. And Yoshua Bengio's Deep Learning book (available online) is almost becoming a classic before it is published. But, you need a few of those books in order to build a somewhat comprehensive and balanced understanding of the field.

I am designing a neural network in Lasagne, a Theano based Deep Learning Library. I am trying to program a simple, Reinforcement Learning network, but am running into a road block in defining the loss function. Basically, the input can be thought of as a location of the AI. The AI needs to get closer to a fixed destination point. The distance can be calculated by the input alone.

Check out the Github repo for an implementation of TD-Gammon with TensorFlow. A few weeks ago AlphaGo won a historic tournament playing the game of Go against Lee Sedol, one of the top Go players in the world. Many people have compared AlphaGo to DeepBlue, which won a series of famous chess matches against Gary Kasparov, but a different comparison may be made for the game of backgammon. Before DeepMind tackled playing Atari games or built AlphaGo there was TD-Gammon, the first algorithm to reach an expert level of play in backgammon. Gerald Tesauro published his paper in 1992 describing TD-Gammon as a neural network trained with reinforcement learning.

In our view reinforcement learning is going to be as big as deep learning in the next two or three years.

Being able to reason in an environment with a large number of discrete actions is essential to bringing reinforcement learning to a larger class of problems. Recommender systems, industrial plants and language models are only some of the many real-world tasks involving large numbers of discrete actions for which current methods are difficult or even often impossible to apply. An ability to generalize over the set of actions as well as sub-linear complexity relative to the size of the set are both necessary to handle such tasks. Current approaches are not able to provide both of these, which motivates the work in this paper. Our proposed approach leverages prior information about the actions to embed them in a continuous space upon which it can generalize. Additionally, approximate nearest-neighbor methods allow for logarithmic-time lookup complexity relative to the number of actions, which is necessary for time-wise tractable training. This combined approach allows reinforcement learning methods to be applied to large-scale learning problems previously intractable with current methods. We demonstrate our algorithm's abilities on a series of tasks having up to one million actions.

Poker playing artificial intelligence has already "approached the performance" of human experts and can use "state-of-the-art methods" in its gameplay. Researchers from University College London - including a staff member from DeepMind's Go defeating team - have created a series of reinforcement algorithms that are able to play Texas Hold'em and a simplistic Leduc poker. The AI is able to learn the game without any prior knowledge of strategies and taught itself by playing fictitious matches on its own, according to the paper Deep Reinforcement Learning from Self-Play in Imperfect-Information Games. Research student Johannes Heinrich and lecturer and David Silver explain in the paper that the Neural Fictitious Self-Play method they created used deep reinforcement learning "to learn directly from their experience of interacting in the game". The method learnt from its mistakes and developed ways to win the games, while also utilising neural networks.

Lately I've been exploring reinforcement learning. I built a q-learning agent for Othello. It is table-based, so it obviously doesn't work well because the state-space of Othello is just too big for a table. So for the past week I've been investigating neural networks as q-learning approximators. I even put one together using Keras/Theanos, and something's going right because it wins 90% of games against an opponent that plays purely randomly (but gets crushed against monte-carlo, another agent I wrote, even if the simulation time for MC is very very short).

Off-policy reinforcement learning has many applications including: learning from demonstration, learning multiple goal seeking policies in parallel, and representing predictive knowledge. Recently there has been an proliferation of new policy-evaluation algorithms that fill a longstanding algorithmic void in reinforcement learning: combining robustness to off-policy sampling, function approximation, linear complexity, and temporal difference (TD) updates. This paper contains two main contributions. First, we derive two new hybrid TD policy-evaluation algorithms, which fill a gap in this collection of algorithms. Second, we perform an empirical comparison to elicit which of these new linear TD methods should be preferred in different situations, and make concrete suggestions about practical use.

Help with Machine Learning and Elevators project (self.MachineLearning) I am beginning to develop a paper about improving elevators waiting times with machine learning, but i am relatively new on the subject (undergrad). I am thinking on using Reinforcement Learning, perhaps along with TensorFlow?