Bellman equation explained

In this article, I am going to explain the Bellman equation, which is one of the fundamental elements of reinforcement learning. The equation tells us what long-term reward can we expect, given the state we are in and assuming that we take the best possible action now and at each subsequent step. Obviously, the goal of reinforcement learning is to maximize the long-term reward, so the Bellman equation can be used to calculate whether we have achieved the goal. The equation below is the Bellman equation for deterministic environments. It gives the value of the current state when the best possible action is chosen in this (and all following steps).