Several approaches to predictive uncertainty quantification have been introduced in recent years, considering uncertainty inaBayesian sense [2,7,18]aswell asfrom afrequentist'spoint ofview [15,5,34].

Note that, because the number of basis vectors parameterized by62 the imaginary coefficients is smaller,there are four gaps in Fig. S2. The results ofthis experiment are illustrated inFig. Because the eigendecomposition ofID is isotropic, we can see that the logistic regression has no159 directional bias.160 8 Example3(Single hidden-layer neural network). Surprisingly, both algorithms yield very similar results, but the algorithm based on the191 eigendecomposition ofthegradient covariance isnumerically much more stable. Meanwhile, thegradient covariance onlyrequires information about firstorder gradients195 and these are orders of magnitudes larger than the second derivatives.

In machine learning, given a finite set of samples, there are usually multiple solutions that can perfectly fit the training data, but theinductive biasof a learning algorithm selects and prioritizes those solutions that agree with itsaprioriassumptions [1,2].

The issue of distribution shifts is emerging as a critical concern in graph representation learning.

In this paper, we propose a novel metric calledcompression ratioto capture the effect of PCA on high-dimensional noisy data.