Asia
AlleviatingtheSampleSelectionBiasinFew-shot LearningbyRemovingProjectiontotheCentroid
While agood feature extractor may help cluster unseen data, thetask distribution shift between training andtesting [25] still makes it hard to estimate novel class distribution using a small number of samples from the support set. Thus, the performance is strongly correlated with the sample quality of the support data.
Laplacian Autoencoders for Learning Stochastic Representations
Established methods for unsupervised representation learning such as variational autoencoders produce none or poorly calibrated uncertainty estimates making it difficult to evaluate if learned representations are stable and reliable. In this work, we present a Bayesian autoencoder for unsupervised representation learning, which is trained using a novel variational lower bound of the autoencoder evidence. This is maximized using Monte Carlo EM with a variational distribution that takes the shape of a Laplace approximation. We develop a new Hessian approximation that scales linearly with data size allowing us to model high-dimensional data. Empirically, we show that our Laplacian autoencoder estimates well-calibrated uncertainties in both latent and output space. We demonstrate that this results in improved performance across a multitude of downstream tasks.
NeuronwithSteadyResponseLeadstoBetter Generalization
Because the deep learning models for the classification task always have a normalization operation (e.g., Softmax) to make the final unconstrained According to the definition of the Consistency of Representations Complexity Measure Eq.(??), itiseasy toseethatwehaveaninfinite number oflocal minima where themeasuresSi andMi,j for arbitrary classesiandj are finite positivenumbers. The python libraries we use to implement our experiments are PyTorch1.7.1andPyG1.6.3. C.2 DetailsofBaselineMethods In this subsection, we detailed the network architectures and the baseline methods used in our experiments. ImageNet is a benchmark dataset used for ResNet-50, which contains 14,197,122 annotated images with 1000 classes. For GNN, we selected four real-world graph datasets: PubMed [11] is a paper citation network where nodes represent documents and edgesrepresentcitationlinks.