Revisiting and Benchmarking Graph Autoencoders: A Contrastive Learning Perspective

Graph autoencoders (GAEs) are self-supervised learning models that can learn meaningful representations of graph-structured data by reconstructing the input graph from a low-dimensional latent space. Over the past few years, GAEs have gained significant attention in academia and industry. In particular, the recent advent of GAEs with masked autoencoding schemes marks a significant advancement in graph self-supervised learning research. While numerous GAEs have been proposed, the underlying mechanisms of GAEs are not well understood, and a comprehensive benchmark for GAEs is still lacking. We revisit the GAEs studied in previous works and demonstrate how contrastive learning principles can be applied to GAEs. Motivated by these insights, we introduce lrGAE (left-right GAE), a general and powerful GAE framework that leverages contrastive learning principles to learn meaningful representations. Our proposed lrGAE not only facilitates a deeper understanding of GAEs but also sets a new benchmark for GAEs across diverse graph-based learning tasks. In the last years, self-supervised learning (SSL) has emerged as a powerful learning paradigm for learning graph representations, approaching, and sometimes even surpassing, the performance of supervised counterparts on many downstream tasks Hjelm et al. (2019); van den Oord et al. (2018). Compared with supervised learning, self-supervised learning gets equal or even better performance with limited or no-labeled data which saves much annotation time and plenty of resources. In a nutshell, SSL purely makes use of rich unlabeled data via well-designed pretext tasks that exploit the underlying structure and patterns in the data. Most recent approaches are shaped by the design of pretext tasks and architectural design, which has led to two lines of research: contrastive and non-contrastive learning Garrido et al. (2023); Balestriero & LeCun (2022). As one of the most successful and widespread SSL strategies, contrastive learning has first shown promising performance in vision representation learning Chen et al. (2020); Gao et al. (2021). It brings together embeddings of different views of the same image while pushing away the embeddings from different ones. Contrastive learning develops rapidly and has recently been applied to the graph learning domain because of the scarcity of graph datasets with labels.