Supplementarymaterial: NeuralAnisotropy Directions AnonymousAuthor(s) Affiliation Address email

Note that, because the number of basis vectors parameterized by62 the imaginary coefficients is smaller,there are four gaps in Fig. S2. The results ofthis experiment are illustrated inFig. Because the eigendecomposition ofID is isotropic, we can see that the logistic regression has no159 directional bias.160 8 Example3(Single hidden-layer neural network). Surprisingly, both algorithms yield very similar results, but the algorithm based on the191 eigendecomposition ofthegradient covariance isnumerically much more stable. Meanwhile, thegradient covariance onlyrequires information about firstorder gradients195 and these are orders of magnitudes larger than the second derivatives.