Organization In Section A, we first introduce the baselines and our model and then describe the experimental details of graph classification and link prediction tasks but also our in-depth analyses. Then, in Section B, we provide the additional experimental results about analyses on datasets, ablation study for our proposed objectives, effects of our hyperparameters (λ1, α, λ2, and the perturbation magnitude), ablation study of attribute masking, and the comparison with augmentation-freeapproaches. In particular,thepre-training dataset consists of306K unlabeled protein ego-networksof50species,andthe fine-tuning dataset consists of 88K protein ego-networks of 8 species with the label given by the functionalityoftheegoprotein. For pre-training, the number of epochs is 100, the batch size is128, the learning rate is0.001, and the margin is10. For fine-tuning, we also follow the conventional setting from Hu et al.[3]. ForJOAOandGraphLoG, we use the publicsource codes4,toobtain the pre-trained models.

Figure 1: (a) Conventional predictive learningthat aims to predict local attributes by masking them.(b) Conventional contrastive learningthat could maximize the similarity of dissimilar graphs perturbed from original graphs.

Rashomon sets can be large in size and complicated in structure, particularly for highly nonlinear function classes that allow complexinteraction terms, such asdecision trees.

The number of cleaned counter-examples across data sets and models is more than 30% of the total number of cleaned examples. FIM-based approaches outperform the LISSA estimator. FIM, which is difficult to store and invert. Figure 3 shows the results of the evaluation of Top Fisher, Practical Fisher and nearest neighbor (NN). As reported in the main text, Practical Fisher lags behind Top Fisher in all cases.

We tackle sequential learning under label noise in applications where a human supervisor can be queried to relabel suspicious examples.

With1 , Err(bY(bT)) =O Vd log (n /d) n + log ( 1/ ) n .

This same algorithm can then be used to reconstruct the true DAGGfrom the true ordering . Once the ordering is known, existing nonlinear variable selection methods [4, 11, 16, 25, 28, 46] suffice to learn the parent setspa(j)and hence the graphG. In our experiments, we use exactly this procedure to learnGfrom the order, based on the data. There are two cases: (i)Bj =, and (ii)Bj 6= . If instead we haveσ23 = var(z3) = 1/3, the condition would be violated.