Optimal Approximation and Learning Rates for Deep Convolutional Neural Networks

One of the most important reasons for its success is the architecture (or structure) [2] which autonomously encodes the a-priori information in the network and significantly reduces the number of free parameters simultaneously to improve the learning performance. Deep convolutional neural networks (DCNNs), a widely used structured deep neural networks, have been triggered enormous research activities in both applications [3, 4, 5] and theoretical analysis [6, 7, 8]. In this paper, we focus on approximation and learning performance analysis for DC-NNs induced by the rectifier linear unit (ReLU) σ(t) = max{t,0}.