These findings enable us to derive an analytical Expectation-Maximization (EM)algorithm forgradient-free DGNlearning.

The source codes are implemented withPytorch 1.10.1,and We select four sub-sets as the OOD benchmark, namelyProtozoa, Microorganisms, Plants, andMollusks. Table 2 compares the performance against all thepost hocbaselines. One of the earliest work considered directly using the Maximum Softmax Probability (MSP) as the scoring function for OOD detection. In [19], the authors observed that the activations of the penultimate layer are quite different for ID and OOD data.

Inspite ofitsgood (promising) performance, ratio-cut and other traditional spectral clustering methods (SC) suffer from the following drawbacks: (1) The timecomplexityoftraditional spectral clustering isO(n2c),which isoneofsignificant drawbacks of SC. Much effort has been devoted to accelerate the process.

Problem (1) has been comprehensively investigated in the literature [Duchi et al., 2011, Kingma and Ba, 2015, Loshchilov and Hutter, 2017], and it is well-known that the classical stochastic gradient descent (SGD) achieves a convergence rate of

Thatis,wedemonstrate that a finite variance isnot necessaryfor almost sure convergence of stochastic NPG, while controlling update aggressiveness is both necessary and sufficient.