Asia
Learning
Whiletheseapproaches arewidely used inpractice andachieveimpressiveempirical gains, their theoretical understanding largely lags behind. Towards bridging this gap, we present a unifying perspectivewhere several such approaches can beviewed asimposing a regularization on the representation via alearnable function using unlabeled data. Wepropose adiscriminativetheoretical framework for analyzing the sample complexity of these approaches, which generalizes the framework of [3] to allow learnable regularization functions.
sidedCalibrationTheorem
Theorem 2. Suppose that the predictive distribution Q has the sufficient ability to approximate the true unknown distribution P, given data is i.i.d. Lm(P,Q) = 0 if and only if P = Q when F is a unit ball in a universal RKHS [13]. Becausetheconfidencelevelp2 p1 is exactly equal to the proportion of samples {y1,,yn} covered by the two-sided prediction interval. B.1 Baselines MC-Dropout (MCD) [12]: A variant of standard dropout, named as Monte-Carlo Dropout. Heteroscedastic Neural Network (HNN) [17]: In this approach, similar to a heteroscedastic regression, the network has two outputs in the last layer, corresponding to the predicted mean and variance for each input xi.