Prono, Luciano
Adaptive Robotic Arm Control with a Spiking Recurrent Neural Network on a Digital Accelerator
Linares-Barranco, Alejandro, Prono, Luciano, Lengenstein, Robert, Indiveri, Giacomo, Frenkel, Charlotte
With the rise of artificial intelligence, neural network simulations of biological neuron models are being explored to reduce the footprint of learning and inference in resource-constrained task scenarios. A mainstream type of such networks are spiking neural networks (SNNs) based on simplified Integrate and Fire models for which several hardware accelerators have emerged. Among them, the ReckOn chip was introduced as a recurrent SNN allowing for both online training and execution of tasks based on arbitrary sensory modalities, demonstrated for vision, audition, and navigation. As a fully digital and open-source chip, we adapted ReckOn to be implemented on a Xilinx Multiprocessor System on Chip system (MPSoC), facilitating its deployment in embedded systems and increasing the setup flexibility. We present an overview of the system, and a Python framework to use it on a Pynq ZU platform. We validate the architecture and implementation in the new scenario of robotic arm control, and show how the simulated accuracy is preserved with a peak performance of 3.8M events processed per second.
Event-Based Eye Tracking. AIS 2024 Challenge Survey
Wang, Zuowen, Gao, Chang, Wu, Zongwei, Conde, Marcos V., Timofte, Radu, Liu, Shih-Chii, Chen, Qinyu, Zha, Zheng-jun, Zhai, Wei, Han, Han, Liao, Bohao, Wu, Yuliang, Wan, Zengyu, Wang, Zhong, Cao, Yang, Tan, Ganchao, Chen, Jinze, Pei, Yan Ru, Brüers, Sasskia, Crouzet, Sébastien, McLelland, Douglas, Coenen, Oliver, Zhang, Baoheng, Gao, Yizhao, Li, Jingyuan, So, Hayden Kwok-Hay, Bich, Philippe, Boretti, Chiara, Prono, Luciano, Lică, Mircea, Dinucu-Jianu, David, Grîu, Cătălin, Lin, Xiaopeng, Ren, Hongwei, Cheng, Bojun, Zhang, Xinan, Vial, Valentin, Yezzi, Anthony, Tsai, James
This survey reviews the AIS 2024 Event-Based Eye Tracking (EET) Challenge. The task of the challenge focuses on processing eye movement recorded with event cameras and predicting the pupil center of the eye. The challenge emphasizes efficient eye tracking with event cameras to achieve good task accuracy and efficiency trade-off. During the challenge period, 38 participants registered for the Kaggle competition, and 8 teams submitted a challenge factsheet. The novel and diverse methods from the submitted factsheets are reviewed and analyzed in this survey to advance future event-based eye tracking research.