Deep Learning
Random deep neural networks are biased towards simple functions
Giacomo De Palma, Bobak Kiani, Seth Lloyd
Neural Information Processing Systems http://nips.cc/
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced
Simon S. Du, Wei Hu, Jason D. Lee
Neural Information Processing Systems http://nips.cc/
Zero-shot Knowledge Transfer via Adversarial Belief Matching
Paul Micaelli, Amos J. Storkey
However,duetogrowing dataset sizes and stricter privacy regulations, it is increasingly common not to have access to the data that was used to train the teacher. We propose a novel method which trains a student to match the predictions of its teacher without using anydata ormetadata. Weachievethisbytraining anadversarial generator to search for images on which the student poorly matches the teacher, and then using them to train the student.
A Benchmark for Interpretability Methods in Deep Neural Networks
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim
Neural Information Processing Systems http://nips.cc/
Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning
Rui Luo, Jianhong Wang, Yaodong Yang, Jun WANG, Zhanxing Zhu
Neural Information Processing Systems http://nips.cc/
NeuralFuse: Learning to Recover the Accuracy of Access-Limited Neural Network Inference in Low-Voltage Regimes Hao-Lun Sun
Energy-efficient computing is of primary importance to the effective deployment of deep neural networks (DNNs), particularly in edge devices and in on-chip AI systems. Increasing DNN computation's energy efficiency and lowering its carbon footprint require iterative efforts from both chip designers and algorithm developers.