) for fully connected networks trained on MNIST vs. depth

We thank the reviewers for the detailed and insightful reviews. We answer most of the questions and will incorporate the feedbacks into the final version. Right: Log leading terms for spectral vs. our bound on WideResNet trained on CIFAR10 using different depths. In Figure 1, we address questions about empirical evaluation of our bounds. The primary challenge is that Theorem 5.1 requires the augmented indicators on the Jacobian norms to be themselves Lipschitz w.r.t. the hidden layers.