Deep Learning
Recurrent Space-time Graph Neural Networks
Andrei Nicolicioiu, Iulia Duta, Marius Leordeanu
It is not yet well understood how to integrate information in space and time into a single, general model.
Evaluating Protein Transfer Learning with TAPE
Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Peter Chen, John Canny, Pieter Abbeel, Yun Song
Machine learning applied to protein sequences is an increasingly popular areaof research.
Implicit Generation and Modeling with Energy Based Models
Neural Information Processing Systems http://nips.cc/
Towards Exact Gradient-based Training on Analog In-memory Computing
While inference on analog accelerators has been studied recently, the training perspective is underexplored. Recent studies have shown that the "workhorse" of digital AI training - stochastic gradient descent (
Stochastic Shared Embeddings: Data-driven Regularization of Embedding Layers
Liwei Wu, Shuqing Li, Cho-Jui Hsieh, James L. Sharpnack
Neural Information Processing Systems http://nips.cc/
Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reduction
The recently proposed stochastic Polyak stepsize (SPS) and stochastic line-search (SLS) for SGD have shown remarkable effectiveness when training over-parameterized models. However, two issues remain unsolved in this line of work.