The Neuromorphic Memory Landscape for Deep Learning

There are several competing processor efforts targeting deep learning training and inference but even for these specialized devices, the old performance ghosts found in other areas haunt machine learning as well. Some believe that the way around the specter of Moore's Law as well as Dennard scaling and data movement limitations is to start thinking outside of standard chip design and look to the human brain for inspiration. This idea has been tested out with various brain-inspired computing devices, including IBM's TrueNorth chips among others over the years. However, deep learning presents new challenges architecturally and in terms of software development for such devices. With GPUs dominating in neural network training in particular (with other devices vying to tackle training and inference on the same device), the question is where brain-inspired approaches might fit in the market and what advantages they might have over GPUs, even in theory (or software, for that matter).