Enhancing Semi-supervised Learning with Noisy Zero-shot Pseudolabels

The growing scale of machine learning applications has made data labeling costs a critical bottleneck in deploying ML systems [1, 2, 3]. Semi-supervised learning (SSL) addresses this challenge by leveraging unlabeled data alongside limited labeled examples [4]. Traditional SSL approaches like pseudo-labeling and consistency regularization have demonstrated strong performance across domains, particularly in computer vision and natural language processing [5, 6, 4]. Recent advances in foundation models have enabled zero-shot inference on novel tasks without taskspecific training [7, 8]. These models can generate predictions for unseen tasks by leveraging their pretrained knowledge, offering a promising direction for reducing labeling requirements. Several works have proposed integrating these zero-shot capabilities into SSL frameworks [9, 10]. Current approaches primarily use foundation models as teacher networks for generating pseudo-labels through inference, which requires complex model distillation and introduces additional training overhead.