Asymptotic convexity of wide and shallow neural networks

--For a simple model of shallow and wide neural networks, we show that the epigraph of its input-output map as a function of the network parameters approximates epigraph of a. convex function in a precise sense. This leads to a plausible explanation of their observed good performance. There has been considerable interest in analyzing the observed empirical success of wide neural networks, both shallow and deep. A small sample of the enormous activity in this domain can be found in [2], [3], [5]-[8], [10], [16], [17]. While the latter property is a consequence of convexity, it does not imply convexity and it may begin to hold for neural networks with'sufficiently large width'.