Neural network approximation and estimation of classifiers with classification boundary in a Barron class

This article concerns the approximation and statistical estimation of high-dimensional, discontinuous functions by neural networks. More precisely, we study a certain class of target functions for classification problems, such as those encountered when automatically labeling images. For such problems, deep learning methods--based on the training of deep neural networks with gradient-based methods--achieve state of the art performance [32, 34]. The underlying functional relationship of such an (image) classification task is typically extremely high-dimensional. For example, the most widely used image databases used to benchmark classification algorithms are MNIST [35] with 28 28 pixels per image, CIFAR-10/CIFAR-100 [31] with 32 32 pixels per image and ImageNet [14, 32] which contains high-resolution images that are typically down-sampled to 256 256 pixels. Compared to practical applications, these benchmark datasets are relatively low-dimensional. Yet, already for MNIST, the simplest of those databases, the input dimension for the classification function is d 784. It is well known in classical approximation theory that high-dimensional approximation problems typically suffer from the so-called curse of dimensionality [11,40].