Reviews: Principles of Riemannian Geometry in Neural Networks

The paper develops a mathematical framework for working with neural network representations in the context of finite differences and differential geometry. In this framework, data points going though layers have fixed coordinates but space is smoothly curved with each layer. The paper presents a very interesting framework for working with neural network representations, especially in the case of residual networks. Unfortunately, taking the limit as the number of layers goes to infinity does not make practical application very easy and somewhat limits the impact of this paper. The paper is not always completely clear.