Despite its popularity, self-training is well-believed to be unreliable and often leads to training instability.

However, the majority of work on self-training has focused on the objective of improving accuracy whereas practical machine learning systems can havecomplex goals (e.g.

Algorithm M-Adam todenote Require: 2R>0: stepsize Require: 1, 2 2[0,1): exponential itssquare Require: : afixedsmall Require:L( ): Astochastic .

Inthemain6 objective, the program optimizes forΛ based on the supervised loss of the "validation" set. SSL typically uses an'unsupervised loss' to15 leverage unlabeled data. While the model may not generalize if the unsupervised loss is poorly designed, recent16 works [38, 36] empirically validate their proposed loss. Theoretical analysis of SSL has also been provided under17 various assumptions,e.g., [6, A]. Weencourage R1 to study these works which show how unsupervised losses aid18 generalization.

However, when the size of labeled data is very small (say a few labeled samples per class), SSL performs poorly and unstably, possibly due to the low qualityoflearnedpseudolabels.Inthispaper,weproposeanewSSLmethodcalled DP-SSL that adopts an innovative data programming (DP) scheme to generate probabilistic labels for unlabeled data. Different from existing DP methods that rely on human experts to provide initial labeling functions (LFs), we develop a multiple-choice learning (MCL) based approach to automatically generate LFs fromscratchinSSLstyle. Withthenoisylabelsproduced bytheLFs,wedesign a label model to resolve the conflict and overlap among the noisy labels, and finally infer probabilistic labels for unlabeled samples.