teacher-student paradigm
Polite Teacher: Semi-Supervised Instance Segmentation with Mutual Learning and Pseudo-Label Thresholding
Filipiak, Dominik, Zapała, Andrzej, Tempczyk, Piotr, Fensel, Anna, Cygan, Marek
The proposed architecture relies on the Teacher-Student mutual learning framework. To filter out noisy pseudo-labels, we use confidence thresholding for bounding boxes and mask scoring for masks. The approach has been tested with CenterMask, a single-stage anchor-free detector. To the best of our knowledge, this is one of the first works tackling the problem of semi-supervised instance segmentation and the first one devoted to an anchor-free detector. The advent of deep learning transformed computer vision pipelines both in academia and industry. However, progress is often hindered, since deep learning models are expensive to train for several reasons. Leaving the hardware and computational expenses aside, the vast share of costs often comes from providing the right amount of samples to learn from. For a number of supervised problems in computer vision, it is relatively easy to obtain data. However, labelling them is often the real source of expenses. Semi-supervised learning methods are tailored to deal with the situation in which there are enough data samples, but access to the labels is severely limited.
Distributed Deep Learning under Differential Privacy with the Teacher-Student Paradigm
Zhao, Jun (Carnegie Mellon University, Nanyang Technological University)
The goal of this work in progress is to address distributed deep learning under differential privacy using the teacher-student paradigm. In the setting, there are a number of distributed entities and one aggregator. Each distributed entity leverages deep learning to train a teacher network on sensitive and labeled training data. The knowledge of the teacher networks is transferred to the student network at the aggregator in a privacy-preserving manner that protects the sensitive data. This transfer results from training non-sensitive and unlabeled data. We also apply secure multi-party computation to securely combining the outputs of local machine learning, in order to update a global model.