Reviews: Large Scale Structure of Neural Network Loss Landscapes

This paper takes a unique approach and aims high. In deep learning, there are all these intriguing empirical observations previously known; the road most travelled to understand them is to prove these observations under certain assumptions, while the authors choose to link these observations through a descriptive model that otherwise could have nothing to do with neural networks. I appreciate this unique approach, which is actually a dominant approach in other sciences like physics. The descriptive model in this paper, if more accurate than not, could potentially simplify the conceptual understanding of optimization for deep learning and motivate new algorithms. There are two reasons why I cannot more enthusiastically recommend this paper.