Analysis on the Nonlinear Dynamics of Deep Neural Networks: Topological Entropy and Chaos

It has brought a paradigm shift to artificial intelligence and many cross-disciplinary areas. Despite its great success in applications, the theoretical explanation and mathematical framework are still open problems for DNN. There have been substantial efforts on the mathematical analysis for DNN, such as using the wavelet framework to understand DNN [12], applying the framework of function approximation [10], enumerating the linear regions after the nonlinear mapping of DNN [14] and analyzing the shape deformation in the transient chaos of DNN [15]. Although these excellent studies have made significant progress on a deeper understanding of DNN, they are still insufficient to fully describe the behavior, quantitatively analyze the performance and thus systematically design DNNs. In this paper, we propose a systematic framework to analyze DNN by considering it as a dynamical system.