We investigate the spectral properties of linear-width feed-forward neural networks, where the sample size is asymptotically proportional to network width.
This framework has the flexibility of deriving distribution-and algorithm-dependent bounds, which are often tighter than VC-related uniform convergence bounds.