Statistical Learning
Time-Independent Information-Theoretic Generalization Bounds for SGLD
We provide novel information-theoretic generalization bounds for stochastic gradient Langevin dynamics (SGLD) under the assumptions of smoothness and dissi-pativity, which are widely used in sampling and non-convex optimization studies.
Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures
In this paper, we study this "benign
A.1 ThePólya-Gammaaugmentation A random variableω has a Pólya-Gamma distribution if it can be written as an infinite sum of independentgammarandomvariables: ω D = 1 2π2 X
GivenatrainingdatasetD =(X,y)offeaturesandcorresponding labels from {1, ..., T} classes,D is partitioned recursively to two subsets, according to classes, at each tree level until reaching leaf nodes with data from only one class. More concretely, initially, feature vectors for all samples are obtained (using a NN), then a class prototype is generated by averaging the feature vectors belonging to the same class for all classes.
Fast Mixingof Stochastic Gradient Descent with Normalizationand Weight Decay
Under 2.1, 2.3, 5.1, 5.2 and 5.3, let x , ( 0) X , ( 0) xinit 2 U forall , >0 for SGD+WD(2) and SDE(3).