Nvidia Lead Details Future Convergence of Supercomputing, Deep Learning

Deep learning could not have developed at the rapid pace it has over the last few years without companion work that has happened on the hardware side in high performance computing. While the applications and requirements for supercomputers versus neural network training are quite different (scalability, programming, etc.) without the rich base of GPU computing, high performance interconnect development, memory, storage, and other benefits from the HPC set, the boom around deep learning would be far quieter. In the midst of this convergence, Marc Hamilton has watched advancements on the HPC side over the years, beginning in the mid-1990s at Sun, where he spent 16 years, before becoming VP of high performance computing at HP. Now the VP of Solutions Architecture and Engineering, he says that there is indeed a perfect storm of technologies intermixing in both deep learning and HPC--and this bodes well for Nvidia's future business at both supercomputing sites and deep learning shops alike. "The reality is that every one of those supercomputing centers is producing data every year, more than they know what to do with, and they have problems they just can't sole with classic scientific computing and HPC approaches," Hamilton tells The Next Platform. "The number of people looking at how to apply deep learning to curing cancer or tackling weather prediction with deep learning is growing. What someone does to optimize a deep learning software package is different than what is needed in HPC, but at the end of the day, it's all matrix math. And that is what we do well; so HPC benefits and machine learning benefits."