Demystifying Oversmoothing in Attention-Based Graph Neural Networks

The latter can be viewed as attention-based GNNs on complete graphs. In this paper, we provide a definitive answer to this question -- attention-based GNNs also lose expressive power exponentially, albeit potentially at a slower exponential rate compared to GCNs.