Abstract: Deep neural networks have achieved great success, most notably in learning to classify images. Yet, the phenomenon of learning images is not well understood, and generalization of deep networks is considered a mystery. Recent studies have explained the generalization of deep networks within the framework of interpolation. In this talk, we will see that the task of classifying images requires extrapolation capability, and interpolation by itself is not adequate to understand deep networks. We study image classification datasets in the pixel space, the internal representations of images learned throughout the layers of trained networks, and also in the low-dimensional feature space that one can derive using wavelets/shearlets.

How does this lead to innovation? What does this have to do with Deep Learning? Intuition like consciousness is something that we are all aware of its existence but likely have not investigated in enough detail to have a grounded understanding of its nature. In fact, I would say that there's more research on the nature of consciousness than research on intuition. I've written earlier about a few research groups that have explored consciousness with respect to an artificial general intelligence, however I don't think has been equivalently the same effort with the study of intuition.