A Note on Connectivity of Sublevel Sets in Deep Learning

Geometry of neural network loss landscape has been studied via the analysis of the global optimality of local minima [1, 2, 5, 8, 9, 14], the existence of a continuous descending path to a global optimum [4, 7, 10, 11, 13], the connectivity of dropout-stable solutions [6, 12], and the topology of sublevel sets [7, 13]. In this paper, we improve the result of [7]. In particular, [7] shows that for a general class of convex loss functions (e.g.