We believe that there exist classic deep learning papers which are worth reading regardless of their application domain. Rather than providing overwhelming amount of papers, We would like to provide a curated list of the awesome deep learning papers which are considered as must-reads in certain research domains. Background Before this list, there exist other awesome deep learning lists, for example, Deep Vision and Awesome Recurrent Neural Networks. Also, after this list comes out, another awesome list for deep learning beginners, called Deep Learning Papers Reading Roadmap, has been created and loved by many deep learning researchers. Although the Roadmap List includes lots of important deep learning papers, it feels overwhelming for me to read them all.

Editor's note: This post originally appeared as an answer to a Quora question, which also included the following: "As I understand, the chance of having a derivative zero in each of the thousands of direction is low. Is there some other reason besides this?" Yes, there is a'theoretical justification', and has taken a couple decades to flush it out. I will first point out, however, it has been observed in practice. This was pointed out by LeCun in his early work on LeNet, and is actually discussed in the'orange book', "Pattern Classification" by David G. Stork, Peter E. Hart, and Richard O. Duda. The problem has been addressed in condensed matter physics 20 years ago in the study of spin glasses.

In the last decade, Data Management personnel solved business problems with data; in the next decade, highly capable machines using Artificial Intelligence Applications will solve problems with available data in a scale unheard before. As the algorithm economy continues to gain momentum among global businesses, the challenges facing Deep Learning are still real. Big Data going mainstream may successfully help combat the Data Management issues making Big Data and Deep Learning the formidable combination for unlocking any complex data handling problem. How Artificial Intelligence is Revolutionizing IT Operation Analytics companies are already leveraging AI-powered Operations Analytics to optimize real-time business operations with "unprecedented granularity, preciseness, and impact."

Computers have transformed work and play, transportation and medicine, entertainment and sports. Yet for all their power, these machines still cannot perform simple tasks that a child can do, such as navigating an unknown room or using a pencil. The solution is finally coming within reach. It will emerge from the intersection of two major pursuits: the reverse engineering of the brain and the burgeoning field of artificial intelligence. Over the next 20 years, these two pursuits will combine to usher in a new epoch of intelligent machines. Why do we need to know how the brain works to build intelligent machines? Although machine-learning techniques such as deep neural networks have recently made impressive gains, they are still a world away from being intelligent, from being able to understand and act in the world the way that we do. The only example of intelligence, of the ability to learn from the world, to plan and to execute, is the brain.

Intel has planted some solid stakes in the ground for the future of deep learning over the last month with its acquisition of deep learning chip startup, Nervana Systems, and most recently, mobile and embedded machine learning company, Movidius. These new pieces will snap into Intel's still-forming puzzle for capturing the supposed billion-plus dollar market ahead for deep learning, which is complemented by its own Knights Mill effort and software optimization work on machine learning codes and tooling. At the same time, just down the coast, Nvidia is firming up the market for its own GPU training and inference chips as well as its own hardware outfitted with the latest Pascal GPUs and requisite deep learning libraries. While Intel's efforts have garnered significant headlines recently with that surprising pair of acquisitions, a move which is pushing Nvidia harder to demonstrate GPU acceleration (thus far the dominant compute engine for model training) for deep learning, they still have some work to do to capture mindshare for this emerging market. Further complicating this is the fact that the last two years have brought a number of newcomers to the field--deep learning chip upstarts touting the idea that general purpose architectures (including GPUs) cannot compare to a low precision, fixed point, specialized approach.

Tensors and new machine learning tools such as TensorFlow are hot topics these days, especially among people looking for ways to dive into deep learning. Turns out, when you look past all the buzz, there's really some fundamentally powerful, useful and usable methods that take advantage of what tensors have to offer, and not just for deep learning situations. If computing can be said to have traditions, then numerical computing using linear algebra is one of the most venerable. Packages like LINPACK and the later LAPACK, are now very old, but are still going strong. At its core, linear algebra consists of fairly simple and very regular operations involving repeated multiplication and addition operations on one- and two-dimensional arrays of numbers (often called vectors and matrices in this context) and it is tremendously general in the sense that many problems can be solved or approximated by linear methods. The absolutely fundamental operation of linear algebra as implemented on computers is the dot product of two vectors.

The technological ecosystem needed to enable AI has finally formed. And, just like a perfect storm, AI's timeline and path is hard to predict and many business owners don't know whether to closely follow and obsess with it or hope it passes them by altogether. What comprises the ecosystem needed to set the growth curve sharply upwards, as in the classic hockey stick analogy we all know very well? First, vast and rich data sets are forming rapidly. Big data require hefty processing power and storage, which is becoming more cost-effective and accessible every day, even to small companies.

Summary: Some observations about new major trends and directions in data science drawn from the Strata Hadoop conference in San Jose last week. This is always exciting, enervating, and exhausting but it remains the single best place to pick up on what's changing in our profession. This conference is on a world tour with four more stops before repeating next year. The New York show is supposed to be a little bigger (hard to imagine) but the San Jose show is closest to our intellectual birthplace. After all this is the place where to call yourself a nerd would be regarded as a humble brag.

Sitting on a stage in Wuzhen, China, a historic city up the river from Shanghai, Google chairman Eric Schmidt described what he called "the age of intelligence." He trumpeted the rise of deep neural networks and other techniques that allow machines to learn tasks largely on their own, either by finding patterns in vast amounts of data or through their own trial and error. At Google, using a sweeping software tool called TensorFlow, engineers have built deep learning systems that can identify faces and objects in photos, recognize commands spoken into smartphones, and translate one language into another. Schmidt called this the biggest technological change of his lifetime. Then he mentioned China's three largest internet companies: Baidu, Tencent, and Alibaba.

Ask Geordie Rose and Suzanne Gildert, co-founders of the startup Kindred, about their company's philosophy, and they'll describe a bold vision of the future: machines with human-level intelligence. Rose says these will be perhaps the most transformative inventions in history -- and they aren't far away. More intriguing than this prediction is Kindred's proposed path for achieving it. Unlike some of the most cash-flush corporations in Silicon Valley, Kindred is focusing not on chatbots or game-playing programs, but on automating physical robots. Gildert, a physicist who conceived Kindred in 2013 while working with Rose at quantum computing company D-Wave, thinks giving AI a physical body is the only way to make real progress toward a true thinking machine.