We introduce a general framework for analyzing learning algorithms based on the notion of self-regularization, which captures implicit complexity control without requiring explicit regularization. This is motivated by previous observations that many algorithms, such as gradient-descent based learning, exhibit implicit regularization. In a nutshell, for a self-regularized algorithm the complexity of the predictor is inherently controlled by that of the simplest comparator achieving the same empirical risk. This framework is sufficiently rich to cover both classical regularized empirical risk minimization and gradient descent. Building on self-regularization, we provide a thorough statistical analysis of such algorithms including minmax-optimal rates, where it suffices to show that the algorithm is self-regularized -- all further requirements stem from the learning problem itself. Finally, we discuss the problem of data-dependent hyperparameter selection, providing a general result which yields minmax-optimal rates up to a double logarithmic factor and covers data-driven early stopping for RKHS-based gradient descent.

Following this thought, we recast virtual screening as an information retrieval task, i.e., given a

Organization In Section A, we first introduce the baselines and our model and then describe the experimental details of graph classification and link prediction tasks but also our in-depth analyses. Then, in Section B, we provide the additional experimental results about analyses on datasets, ablation study for our proposed objectives, effects of our hyperparameters (λ1, α, λ2, and the perturbation magnitude), ablation study of attribute masking, and the comparison with augmentation-freeapproaches. In particular,thepre-training dataset consists of306K unlabeled protein ego-networksof50species,andthe fine-tuning dataset consists of 88K protein ego-networks of 8 species with the label given by the functionalityoftheegoprotein. For pre-training, the number of epochs is 100, the batch size is128, the learning rate is0.001, and the margin is10. For fine-tuning, we also follow the conventional setting from Hu et al.[3]. ForJOAOandGraphLoG, we use the publicsource codes4,toobtain the pre-trained models.

Subseasonal forecasting of the weather two to six weeks in advance is critical for resource allocation and advance disaster notice but poses many challenges for the forecasting community.

Wevalidate ourmethod, RobustContrastiveLearning(RoCL),onmultiplebenchmarkdatasets, on which itobtains comparable robust accuracyover state-of-the-art supervised adversarial learning methods, and significantly improved robustness against the black boxand unseen types of attacks.