Goto

Collaborating Authors

 Wallonia





How a student becomes a teacher: learning and forgetting through Spectral methods

Neural Information Processing Systems

The above scheme proves particularly relevant when the student network is overparameterized (namely, when larger layer sizes are employed) as compared to the underlying teacher network. Under these operating conditions, it is tempting to speculate that the student ability to handle the given task could be eventually stored in a sub-portion of the whole network.


How a student becomes a teacher: learning and forgetting through Spectral methods

Neural Information Processing Systems

The above scheme proves particularly relevant when the student network is overparameterized (namely, when larger layer sizes are employed) as compared to the underlying teacher network. Under these operating conditions, it is tempting to speculate that the student ability to handle the given task could be eventually stored in a sub-portion of the whole network.





neurips_2021_main

Pedro Luiz Coelho Rodrigues

Neural Information Processing Systems

Made: Maskedautoencoder fordistributionestimation. In Francis Bachand David Blei, editors,Proceedingsofthe 32nd International Conferenceon Machine Learning, volume 37 ofProceedingsof Machine Learning Research, pages 881-889, Lille, France, 07-09 Jul 2015.