Drawing on the intricate structures of the brain, Spiking Neural Networks (SNNs) emerge as a transformative development in artificial intelligence, closely emulating the complex dynamics of biological neural networks. While SNNs show promising efficiency on specialized sparse-computational hardware, their practical training often relies on conventional GPUs. This reliance frequently leads to extended computation times when contrasted with traditional Artificial Neural Networks (ANNs), presenting significant hurdles for advancing SNN research. To navigate this challenge, we present a novel temporal fusion method, specifically designed to expedite the propagation dynamics of SNNs on GPU platforms, which serves as an enhancement to the current significant approaches for handling deep learning tasks with SNNs. This method underwent thorough validation through extensive experiments in both authentic training scenarios and idealized conditions, confirming its efficacy and adaptability for single and multi-GPU systems. Benchmarked against various existing SNN libraries/implementations, our method achieved accelerations ranging from 5 to 40 on NVIDIA A100 GPUs.

Recently, artificial neural networks (ANNs), such as convolutional neural networks (CNNs)[1], recurrent neural networks (RNNs)[2] and transformers[3], have defeated most other methods and even surpassed the average ability levels of humans in some areas, including image classification [1, 4, 5], object detection [6, 7, 8], machine translation [9, 10, 11, 3], speech recognition [12, 13], and gaming [14, 15]. These achievements are computer-science-oriented because ANNs are mainly driven by gradient-based numerical optimization methods[16, 17], big data[18, 19] and massively parallel computing with graphics processing units (GPUs) [20, 21]. Although neuroscience plays a diminished role in ANNs[22], insights from neuroscience are critical for building general human-level artificial intelligence (AI) systems [23, 24]. The human brain is one of the most intelligent systems, possessing overwhelming advantages over any other artificial system in cognition and learning tasks such as transfer learning and continual learning[24]. The neuroscientific community has been exploring biologically plausible computational paradigms to understand, mimic, and exploit the impressive feats of the human brain.