Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis Wuyang Chen University of Texas at Austin Wei Huang

In this work, we theoretically characterize the impact of connectivity patterns on the convergence of DNNs under gradient descent training in fine granularity. By analyzing a wide network's Neural Network Gaussian Process (NNGP), we are able to depict how the spectrum of an