Zero-shot Knowledge Transfer via Adversarial Belief Matching

However,duetogrowing dataset sizes and stricter privacy regulations, it is increasingly common not to have access to the data that was used to train the teacher. We propose a novel method which trains a student to match the predictions of its teacher without using anydata ormetadata. Weachievethisbytraining anadversarial generator to search for images on which the student poorly matches the teacher, and then using them to train the student.