Optimal Nonparametric Inference via Deep Neural Network

For instance, deep neural networks have led to impressive performance in fields such as computer vision, natural language processing, image/speech/audio recognition,social network filtering, machine translation, bioinformatics, drug design, medical image analysis, where they have demonstrated superior performance to human experts. The success of deep networks hinges on their rich expressiveness (see Delalleau and Bengio (2011), Raghu et al. (2017), Montufar et al. (2014), Bianchini and Scarselli (2014), Telgarsky (2016), Liang and Srikant (2017) and Yarotsky (2017, 2018)). Recently, deep networks have played an increasingly important role in statistics particularly in nonparametric curve fitting (see Kohler and Krzyżak (2005); Hamers and Kohler (2006); Kohler and Krzyżak (2017); Kohler and Mehnert (2011);Schmidt-Hieber (2017)). Applications of deep networks in other fields such as image processing or pattern recgnition include, to name a few, LeCun et al. (2015), Deng et al. (2013), Wan et al. (2014), Gal and Ghahramani (2016), etc. A fundamental problem in statistical applications of deep networks is how accurate they can estimate a nonparametric regression function. To describe the problem, let us consider i.i.d.