Supplemental Materialfor " What Neural Networks Memorizeand Why " AProofof Lemma 2.1 Proof.Observethat, bylinearityofexpectation, wehave: inflm(A,S, i,z) = E