ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data

Furthermore, while the neuromorphic community has argued in favor of their higher We seek to enable classic processing of continuous energy efficiency for decades, recent research and breakthroughs ultra-sparse spatiotemporal data generated by in edge AI accelerators suggest that this might not event-based sensors with dense machine learning be the case (Dampfhoffer et al., 2023; Garrett et al., 2023; models. We propose a novel hybrid pipeline composed Moosmann et al., 2023; Caccavella et al., 2023). of asynchronous sensing and synchronous processing that combines several ideas: (1) an embedding Nevertheless, considering the inherent advantages of eventbased based on PointNet models - the ALERT vision sensors, namely high dynamic range (HDR) module - that can continuously integrate new and and high temporal resolution - simultaneously, without any dismiss old events thanks to a leakage mechanism, tradeoffs between the two -, we aim to find a way to leverage (2) a flexible readout of the embedded data this sparse and low-latency data for real-world situations.