A different kind of (deep) learning: part 1

Deep learning has truly reshuffled things in machine learning field, and specifically in image recognition tasks. In 2012, Alex-net has initiated a (still far from ending) race towards solving, or at least significantly improving, computer vision tasks. Each of these research paths improves training quality (speed, accuracy, sometimes generalization), but it seems that doing more of the same thing may result in some gradual improvements, but not a in significant breakthrough. On the other hand, growing body of work in deep learning shows that there are significant flaws in current methods, especially in terms of generalization, e.g this recent one: generalization failure when objects are rotated: So there seems to be a need of improvements that are a bit more aggressive. Or perhaps expanding the research spectrum to ideas that may be a bit riskier.