Our goal is to provide a systematic understanding of the landscape of graph neural networks for knowledge graphs pertaining to the prominent task of link prediction. Our analysis entails a unifying perspective on seemingly unrelated models and unlocks a series of other models.

To compare thequality oftargetsamples being selected fortraining, wemeasure reliability precision (howmanyofthe selected target samples were actually predicted correctly?) We report expected calibration error (ECE [7]), lower is better. We separately visualize features before and after in-domain pretraining with MAE 7and DINO 8. Wenote that these features are completely selfsupervised as the model has not seen task labels yet. Regardless, we observe a small degree of taskdiscriminativeness (examples ofthesame class areclustered together) anddomain invariance (examples of the same class but different domains are close) before additional pretraining. We now measure the degree of label overlap between ImageNet-22K and these 3 benchmarks.

We show it depends on the precise way in which the limit is taken, and in particular on how the quantityofdata,thehiddenlayerwidth,&thelearningratescalesasd .

We initiate the study of stochastic optimization with oblivious noise, broadly generalizing the standard heavy-tailed noise setup.

Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images.