Deep Learning: Generalization Requires Deep Compositional Feature Space Design

Deep learning massive success in almost every fields represents its ability to solve complex problems. The tradeoff between model complexity and accuracy is an important area of deep learning research. Very complex model with millions of parameters [8], [9] proved to the state of the art solution for many vision and natural language problems. A common way to measure the performance or generalizability of a deep learning model is to test it on a well discriminative validation/test set representing the variation of samples of the corresponding problem. Learning very complex model is a matter of the requirements of high computing power and huge dataset.