Goto

Collaborating Authors

 Inductive Learning






PAC-Bayes Analysis Beyond the Usual Bounds

Neural Information Processing Systems

We focus on a stochastic learning model where the learner observes a finite set of training examples and the output of the learning process is a data-dependent distribution over a space of hypotheses. The learned data-dependent distribution is then used to make randomized predictions, and the high-level theme addressed here is guaranteeing the quality of predictions on examples that were not seen during training, i.e. generalization. In this setting the unknown quantity of interest is the expected risk of the data-dependent randomized predictor, for which upper bounds can be derived via a P AC-Bayes analysis, leading to P AC-Bayes bounds. Specifically, we present a basic P AC-Bayes inequality for stochastic kernels, from which one may derive extensions of various known P AC-Bayes bounds as well as novel bounds. We clarify the role of the requirements of fixed'data-free' priors, bounded losses, and i.i.d.



Evaluation Protocol: The most ambitious aim of self-supervised learning is to create universal visual representations

Neural Information Processing Systems

We thank the reviewers for valuable feedback. Before addressing individual comments, we clarify common concerns. Moreover, "image-level" vs "pixel-level" training has no bearing on the validity of evaluating with Any method that uses a CNN learns more than just "image-level" representations; for Our task is to learn pixel-wise semantic-aware embeddings from scratch. We will update the final version to reflect the full 200 training epochs. We first sample regions, then a fixed number of pixels within chosen regions.


In Table A, we repeat our experiments on 5000 test examples for each dataset (or the

Neural Information Processing Systems

We thank all reviewers for their valuable comments and suggestions. Table B highlights our differences. Methods on bottom-left corner are better. We will enlarge figures and explain more. In Table 2 and 3, HIGGS contains 10.5 million training examples and the ensemble We additionally added Bosch (1.2 million examples, 968 features) in Table A. Both datasets are from Our method is effective on both datasets.