Motion Perceiver: Real-Time Occupancy Forecasting for Embedded Systems

Abstract--This work introduces a flexible architecture for realtime occupancy forecasting. In contrast to existing, more computationally expensive architectures, the proposed model exploits recursive latent state estimation, using learned transformer-based prediction and update modules. This allows for highly efficient real-time inference on an embedded system (profiled on an Nvidia Xavier AGX), and the inclusion of a broad set of information from a diverse set of sensors. The above image is color coded as green for true positive (TP), blue as false positive (FP) and red for false Motion forecasting is a critical task for autonomous vehicles negative (FN). We use an occupancy probability > 50% as that need to plan their trajectory in a dynamic environment the threshold to depict a TP or FN detection. This can be alongside other agents. This task is challenging, as the path changed depending on an individual's risk profile. Black is of an agent depends on a variety of aspects, including environmental the rasterized road-graph used as additional context. Moreover, the vast sensor suite available on autonomous vehicles produces data in a variety of structures with different properties.