A Proofs Lemma 1. For the mixed imaged opponent policy (IOP) π