Limitations of Deep Learning and strategic observations

While Deep Learning has shown itself to be very powerful in applications, the underlying theory and mathematics behind it remains obscure and vague. Deep Learning works, but theoretically we do not understand much why it works. Some leading machine learning theorists like Vladimir Vapnik criticise Deep Learning for its ad-hoc approach that gives a strong flavour of brute force rather than technical sophistication. Deep Learning is not theory intensive; it is empirical based more (hence causing battle of viewpoints between empiricism and realism) and relies on clever tweakings [1].[1] This is why'Deep Learning' is viewed as a black box and why we preferred to use Theano instead of other packages as it allowed us better view inside the workings of the model (which is still not enough to fully overcome the black box criticism).