An intro to Advantage Actor Critic methods: let's play Sonic the Hedgehog!

As we saw in the article about improvements in Deep Q Learning, value-based methods have high variability. To reduce this problem, we spoke about using the advantage function instead of the value function. This function will tell us the improvement compared to the average the action taken at that state is. In other words, this function calculates the extra reward I get if I take this action. The extra reward is that beyond the expected value of that state.