WeightedMutualLearningwithDiversity-Driven ModelCompression

Neural Information Processing Systems 

Onlinedistillation collaboratively trains agroup of peer models, which are treated as students, and all students gain extra knowledge from each other.