Analytic Network Learning

Attributed to its high learning capacity with good prediction capability, the deep neural network has found its advantage in wide areas of science and engineering applications. Such an observation has sparked a surge of investigations into the architectural and learning aspects of the deep network for targeted applications. The main ground for realizing the high learning capacity and predictivity comes from several major advancements in the field which include the processing platform, the learning regimen, and the availability of big data size. In terms of the processing platform, the advancement in Graphics Processing Units (GPUs) has facilitated parallel processing of complex network learning within accessible time. Together with the relatively low cost of the hardware, the large number of public high level open source libraries has enabled a crowdsourcing mode of learning architectural exploration. Based on such a learning platform, several learning regimens such as the Convolutional Neural Network (CNN or LeNet-5) [1], the AlexNet [2], the GoogLeNet or Inception [3], the Visual Geometry Group Network (VGG Net) [4], the Residual Network (ResNet) [5] and the DenseNet [6] have stretched the network learning in terms of the network depth and prediction capability way beyond the known boundary established by the conventional statistical methods. 2 Without sufficiently convincing explanation in theory, the advancement of deep learning has been grounded upon'big' data, powerful machinery and crowd efforts to achieve at'breakthrough' results that were not possible before. Such a swarming phenomenon has pushed forward the demand of hardware as well as middle ware, but at the expense of masking the importance of fundamental results available in statistical decision theory. The research scene has arrived at such a state of deeming results unacceptable without working directly on or comparing them with'big' data which implicitly relies on powerful machinery.