LeanResNet: A Low-cost yet Effective Convolutional Residual Networks

Convolutional Neural Networks (CNNs) filter the In recent years there has been an effort to reduce the number input data using a series of spatial convolution of parameters in CNNs. Among the first approaches are operators with compact stencils and point-wise the methods of pruning (Hassibi & Stork, 1992; Han et al., non-linearities. Commonly, the convolution operators 2015; Li et al., 2017) and sparsity (Wen et al., 2016; couple features from all channels, which Changpinyo et al., 2017; Han et al., 2016) that have been leads to immense computational cost in the training typically applied to already trained full networks. It has of and prediction with CNNs. To improve been shown that once a network is trained, a large portion the efficiency of CNNs, we introduce lean convolution of its weights can be removed without hampering its operators that reduce the number of parameters efficiency by much.