2f4b6febe0b70805c3be75e5d6a66918-Supplemental-Conference.pdf

Lemma 3. Let l be any positive integer. There exists anl-layer ReLU networkg with 2n(l 1) hidden neurons and a maximum width of2n such thatg(x) = x for allx Rn. Let the family of closed connected subsets Q = {Xi}i [I] satisfy Assumption 1 for any p Pn,k. Note that everyHlm is an affine subspace of Rn with dimension n 1 (a hyperplane) or an empty set. Because the linear components are distinct, it must be true thatk I by Definition 2. Ifp is an affine function, then it follows that k =min Q Cn,k(p) Q =ϕ(n,k)=1,theclaimholds.Forthenon-affinecase,wemusthave k >1.