A Additional Discussions and Details

A.1 Limitations One potential limitation of DyGFormer lies in the ignorance of high-order relationships between nodes since it solely learns from the first-hop interactions of nodes. In certain scenarios where nodes' high-order relationships are essential, DyGFormer may be suboptimal compared with baselines that learn the higher-order interactions. However, trivially feeding the multi-hop neighbors of nodes into DyGFormer would incur expensive computational costs. It is promising to design more efficient and effective frameworks to model nodes' high-order relationships for dynamic graph learning. Another potential limitation is the sensitivity of the neighbor co-occurrence encoding scheme against different negative sampling strategies (discussed in Section 5.7).