hierarchical max-pooling model
Analysis of convolutional neural network image classifiers in a rotationally symmetric model
Kohler, Michael, Walter, Benjamin
Convolutional neural network image classifiers are defined and the rate of convergence of the misclassification risk of the estimates towards the optimal misclassification risk is analyzed. Here we consider images as random variables with values in some functional space, where we only observe discrete samples as function values on some finite grid. Under suitable structural and smoothness assumptions on the functional a posteriori probability, which includes some kind of symmetry against rotation of subparts of the input image, it is shown that least squares plug-in classifiers based on convolutional neural networks are able to circumvent the curse of dimensionality in binary image classification if we neglect a resolution-dependent error term. The finite sample size behavior of the classifier is analyzed by applying it to simulated and real data.
Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local pooling
Deep learning, i.e., estimation of a functional relationship by a deep neural network, belongs nowadays to the most successful and most widely used methods in machine learning, see, e.g., Schmidhuber (2015) and the literature cited therein. In many applications the most successful networks are deep convolutional networks. E.g., since 2012, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) was won each year by deep convolutional neural networks (cf., Russakovsky et al. (2015)). Deep convolutional neural networks can be considered as a special case of deep feedforward neural networks, where symmetry constraints are imposed on the weights of the networks. For deep feedforward neural networks, recently a large number of impressive rate of convergence results have been shown. E.g., in Kohler and Krzyżak (2017), Bauer and Kohler (2019), Schmidt-Hieber (2020), Kohler and Langer (2021) and Suzuki and Nitanda (2019) it was shown that these networks achieve in nonparametric regression a
On the rate of convergence of image classifiers based on convolutional neural networks
Kohler, M., Krzyzak, A., Walter, B.
Deep neural networks are nowadays among the most successful and most widely used methods in machine learning, see, e.g., Schmidhuber (2015), Rawat and Wang (2017), and the literature cited therein. In many applications the most successful networks are deep convolutional networks, see, e.g., Krizhevsky, Sutskever and Hinton (2012) and Kim (2014) concerning applications in image classification or language recognition, resp. These networks can be considered as a special case of deep feedforward neural networks, where symmetry constraints are imposed on the weights of the networks. For general deep feedforward neural networks it was recently shown that under suitable compository assumptions on the structure of the regression function these networks are able to achieve dimension reduction in estimation of high-dimensional regression functions (cf., Kohler