Deep Learning
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel
Two children spawned later (earlier) than a very early transition time in parent training, will arrive at the same (different) basin on the loss landscape.
Discriminative Topic Modeling with Logistic LDA
Iryna Korshunova, Hanchen Xiong, Mateusz Fedoryszak, Lucas Theis
Neural Information Processing Systems http://nips.cc/
Gradient Surgery for Multi-Task Learning Tianhe Y u
We propose a form of gradient surgery that projects a task's gradient onto the normal plane of the gradient of
On the Power and Limitations of Random Features for Understanding Neural Networks
Considering the one-hidden-layer example above, this corresponds to learning linear predictors over a fixed representation (chosen obliviously and randomly at initialization).
Graph Contrastive Learning with Augmentations Yuning You
We first design four types of graph augmentations to incorporate various priors.
Supplementary Material: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei
Given an input passage and an answer, the task is to generate a question that asks for the answer.
Unsupervised Scalable Representation Learning for Multivariate Time Series
Jean-Yves Franceschi, Aymeric Dieuleveut, Martin Jaggi
Time series constitute a challenging data type for machine learning algorithms, due to their highly variable lengths and sparse labeling in practice.
reviewers; we will make sure to update our manuscript accordingly. 2 1 Comparison with Other Unsupervised Methods (R1)
We would like to thank the reviewers for their comments and suggestions. In particular, TimeNet is a seq2seq method relying on an antoencoding loss and using LSTMs as encoder and decoder. TimeNet, and notably do not scale to long time series (as explained on lines 144-157), unlike ours. However, we did perform experiments on some datasets with different loss variants. We will add insights on this matter to the paper.
Bayesian Bits: Unifying Quantization and Pruning
We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization.