Appendices A Notes on the MDP formulation

Neural Information Processing Systems 

See the complete proof in Appendix C.6.Proof.