Supplementary Material for " Closing the Gap: Tighter Analysis of Alternating Stochastic Gradient Methods for Bilevel Problems "