ProgD: Progressive Multi-scale Decoding with Dynamic Graphs for Joint Multi-agent Motion Forecasting

Accurate motion prediction of surrounding agents is crucial for the safe planning of autonomous vehicles. Recent advancements have extended prediction techniques from individual agents to joint predictions of multiple interacting agents, with various strategies to address complex interactions within future motions of agents. However, these methods overlook the evolving nature of these interactions. To address this limitation, we propose a novel progressive multi-scale decoding strategy, termed ProgD, with the help of dynamic heterogeneous graph-based scenario modeling. In particular, to explicitly and comprehensively capture the evolving social interactions in future scenarios, given their inherent uncertainty, we design a progressive modeling of scenarios with dynamic heterogeneous graphs. With the unfolding of such dynamic heterogeneous graphs, a factorized architecture is designed to process the spatio-temporal dependencies within future scenarios and progressively eliminate uncertainty in future motions of multiple agents. Furthermore, a multi-scale decoding procedure is incorporated to improve on the future scenario modeling and consistent prediction of agents' future motion. Introduction Motion prediction is important for self-driving systems to ensure safe and efficient navigation. Of particular interest is joint multi-agent motion prediction, which involves concurrently forecasting the future trajectories of all agents within a scene. This task has gained increasing attention recently, due to its complexity compared to marginal motion prediction, as it requires maintaining consistency and coherence in future motions of interactive agents, reflecting the intricate dynamics of real-world traffic. Without such consistency, the prediction module could produce conflicting trajectories, such as collisions between the predicted motions of agents, which would undermine the reliability of the system and lead to unsafe or infeasible motion plans. The challenge is further compounded by several intrinsic factors: (1) Dynamic and complex future social interactions.