Tothis coordinate (1), and necessaryand solutionof (1). We usethemodeltrainedusing MixMatch [5] under 3 cases: (1) l = 100, u =1 , (2) = l = u = 100(reverse) and (3) = 100.

However, ifτ is too high, most of the unlabeled data points would not be used for consistency regularization. Based on these insights, we setτ as 0.95 in our experiments. We describe further details of the experimental setup. To train the ReMixMatch, we gradually increased the coefficient of the loss associated with the unlabeled data points, following [4]. We found that without this gradual increase, the validation loss of the ReMixMatch did not converge.

Existing semi-supervised learning (SSL) algorithms typically assume class-balanced datasets, although the class distributions of many real-world datasets are imbalanced.

I cite from ReMixMatch figure caption: "Augmentation anchoring. We use the prediction for a weakly augmented image (green, middle) as the target for predictions on strong augmentations of the same image". This sounds to me as a summary of the presented work, and as such I consider it a special case of the ReMixMatch. Authors have discussed the differences between their work and ReMixMatch, mentioning that (1) "ReMixMatch don t use pseudo labeling", and (2) ReMixMatch uses sharpening of pseudolabels and weight annealing of the unlabeled data loss. However, in section 3.2.1 of ReMixMatch, it is stated that the guessed labels are used as targets (for strongly augmented images) using cross-entropy loss.

While semi-supervised learning (SSL) has proven to be a promising way for leveraging unlabeled data when labeled data is scarce, the existing SSL algorithms typically assume that training class distributions are balanced. However, these SSL algorithms trained under imbalanced class distributions can severely suffer when generalizing to a balanced testing criterion, since they utilize biased pseudo-labels of unlabeled data toward majority classes. To alleviate this issue, we formulate a convex optimization problem to softly refine the pseudo-labels generated from the biased model, and develop a simple algorithm, named Distribution Aligning Refinery of Pseudo-label (DARP) that solves it provably and efficiently. Under various class-imbalanced semi-supervised scenarios, we demonstrate the effectiveness of DARP and its compatibility with state-of-the-art SSL schemes.

A BSTRACT We improve the recently-proposed "MixMatch" semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring. Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels. Augmentation anchoring feeds multiple strongly augmented versions of an input into the model and encourages each output to be close to the prediction for a weakly-augmented version of the same input. To produce strong augmentations, we propose a variant of AutoAugment which learns the augmentation policy while the model is being trained. Our new algorithm, dubbed ReMix-Match, is significantly more data-efficient than prior work, requiring between 5 and 16 less data to reach the same accuracy. For example, on CIFAR-10 with 250 labeled examples we reach 93 .73% This can enable the use of large, powerful models when labeling data is expensive or inconvenient. Research on SSL has produced a diverse collection of approaches, including consistency regularization (Sajjadi et al., 2016; Laine & Aila, 2017) which encourages a model to produce the same prediction when the input is perturbed and entropy minimization (Grandvalet & Bengio, 2005) which encourages the model to output high-confidence predictions. The recently proposed "MixMatch" algorithm (Berthelot et al., 2019) combines these techniques in a unified loss function and achieves strong performance on a variety of image classification benchmarks.