Supplementary Materials for " Multi-Agent Meta-Reinforcement Learning " AT echnical Lemmas