Foothill: A Quasiconvex Regularization Function

Deep learning has recently seen a surge in progress, from training shallow networks to very deep networks consisting of tens to hundreds of layers. Deep neural networks (DNNs) have demonstrated success for many supervised learning tasks (Szegedy et al., 2015; Simonyan and Zisserman, 2014). The focus has been on increasing accuracy, in particular for image, speech, and recently text tasks, where deep convolutional neural networks (CNNs) are applied. The resulting networks often include millions to billions parameters. Having too many parameters, increases the risk of over-fitting and hence a poor model generalization afterall.