Supplementary Materials ATheoretical proofs

Let Z RD and T R d be two random variables that have moments. We first prove the direction Z T SI(Z;T) = 0, which is equivalent to prove I(Z;T) = 0 SI(Z;T) = 0. We prove the contrapositive, i.e. rather than show LHS = RHS, we show that RHS = LHS. This is because for any h,gthat satisfy ρ(h,g) 0, we can always flip the sign of ρ(h,g)by replacing h by h or g by g, so that the value of ρ(h,g)is higher. Z i = [σ(θ i Z)k]Kk=1, T j = [σ(ϕ j T)k]Kk=1, with σ() defined as in the main text l.103. Now assume that supwi,vj ρ(w i Z i,v j T j) > ϵ for some i,j.