Analytic Characterization of the Hessian in Shallow ReLU Models: AT ale of Symmetry

Much of the current effort in understanding the empirical success of artificial neural networks is concerned with the geometry of the associated nonconvex optimization landscapes. Of particular importance is the Hessian spectrum which characterizes the local curvature of the loss at different points in the space.