Scalable NoC-based Neuromorphic Hardware Learning and Inference

Abstract--Bio-inspired neuromorphic hardware is a research direction to approach brain's computational power and energy efficiency. Spiking neural networks (SNN) encode information as sparsely distributed spike trains and employ spike-timingdependent plasticity (STDP) mechanism for learning. Existing hardware implementations of SNN are limited in scale or do not have in-hardware learning capability. In this work, we propose a low-cost scalable Network-on-Chip (NoC) based SNN hardware architecture with fully distributed in-hardware STDP learning capability. All hardware neurons work in parallel and communicate through the NoC. This enables chip-level interconnection, scalability and reconfigurability necessary for deploying different applications. The hardware is applied to learn MNIST digits as an evaluation of its learning capability. We explore the design space to study the tradeoffs between speed, area and energy. How to use this procedure to find optimal architecture configuration is also discussed. In the field of deep learning, convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are developed to perform a series of human-level cognitive applications [1] [2]. However, the tremendous computation and memory requirement have been seriously challenging the processing efficiency of deep learning systems [3] [4]. The limitations of Von Neumann architecture coupled with increasing power demands due to Dennard scaling and the approaching end of Moore's Law have motivated multiple research efforts into low-power, highly parallel and distributed computing architecture [5] [6] [7] [8] and brain-inspired computing architecture [9] [10]. Brain as a source of inspiration is not surprising given its ability to process massive amounts of real-time information while consuming less than 20 W of power [11]. The goal of neuromorphic hardware design is to explore the bio-inspired architecture to achieve cognitive functions in real time utilizing lower power and smaller footprint than the traditional Von Neumann architectures.