Precise neural network computation with imprecise analog devices

The operations used for neural network computation map favorably onto simple analog circuits, which outshine their digital counterparts in terms of compactness and efficiency. Nevertheless, such implementations have been largely supplanted by digital designs, partly because of device mismatch effects due to material and fabrication imperfections. We propose a framework that exploits the power of deep learning to compensate for this mismatch by incorporating the measured device variations as constraints in the neural network training process. This eliminates the need for mismatch minimization strategies and allows circuit complexity and power-consumption to be reduced to a minimum. Our results, based on largescale simulations as well as a prototype VLSI chip implementation indicate a processing efficiency comparable to current state-of-art digital implementations. This method is suitable for future technology based on nanodevices with large variability, such as memristive arrays. The growing need for computing power has led to the exploration of computing technologies beyond the predominant von Neumann architecture. In particular, due to the separation of memory and processing elements, traditional computing systems experience a bottleneck when dealing with problems involving great amounts of high-dimensional data [4, 25], such as image processing, probabilistic inference, or speech recognition. These problems are often best tackled by conceptually simple but powerful and highly parallel models, such as deep neural networks (DNNs), which have delivered state-of-the-art performance on exactly those applications [29]. The fact that DNNs are characterized by stereotypical and simple operations at each unit, which often can be performed in parallel, makes them compatible with the processing style of graphics processing units (GPUs) [46]. The large computational demands of DNNs have simultaneously sparked interest in methods that make neural network inference faster and more power efficient, whether through new algorithmic inventions [19, 22, 12], dedicated digital hardware implementations [6, 17, 9, 34], or by taking inspiration from real nervous systems [15, 38, 33, 24, 40].