UnsupervisedRepresentationTransferforSmall Networks: IBelieveICanDistillOn-the-Fly

Neural Information Processing Systems 

Foreffectiveknowledge transfer,weadopt the idea of domain classifier so that student training is guided by discriminative features invariant totherepresentational space shift between teacher andstudent.