Heterogeneous Interaction Modeling With Reduced Accumulated Error for Multi-Agent Trajectory Prediction