Deep Learning
Preconditioned Spectral Descent for Deep Learning
David E. Carlson, Edo Collins, Ya-Ping Hsieh, Lawrence Carin, Volkan Cevher
Neural Information Processing Systems http://nips.cc/
Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation
Seunghoon Hong, Hyeonwoo Noh, Bohyung Han
Neural Information Processing Systems http://nips.cc/
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective
Large width limits have been a recent focus of deep learning research: modulo computational practicalities, do wider networks outperform narrower ones?
Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks
Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, Yang Liu
Neural Information Processing Systems http://nips.cc/
Learning Physical Constraints with Neural Projections Shuqi Yang
We propose a new family of neural networks to predict the behaviors of physical systems by learning their underpinning constraints.
AGEM: Solving Linear Inverse Problems via Deep Priors and Sampling
Bichuan Guo, Yuxing Han, Jiangtao Wen
Neural Information Processing Systems http://nips.cc/
Non-asymptotic Analysis of Stochastic Methods for Non-Smooth Non-Convex Regularized Problems
Neural Information Processing Systems http://nips.cc/
Skip-Thought Vectors
Ryan Kiros, Yukun Zhu, Russ R. Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, Sanja Fidler
We describe an approach for unsupervised learning of a generic, distributed sentence encoder. Using the continuity of text from books, we train an encoder-decoder model that tries to reconstruct the surrounding sentences of an encoded passage. Sentences that share semantic and syntactic properties are thus mapped to similar vector representations. We next introduce a simple vocabulary expansion method to encode words that were not seen as part of training, allowing us to expand our vocabulary to a million words. After training our model, we extract and evaluate our vectors with linear models on 8 tasks: semantic relatedness, paraphrase detection, image-sentence ranking, question-type classification and 4 benchmark sentiment and subjectivity datasets. The end result is an off-the-shelf encoder that can produce highly generic sentence representations that are robust and perform well in practice.
Why AI Companies Are Pivoting to Short-Form Video
OpenAI's new short-form video app, Sora, seems to have all the ingredients of a viral hit. Just hours after the app's launch on Tuesday, memes created using its AI video-generation technology were already spreading to other social networks--including, for example, a video of OpenAI CEO Sam Altman rapping from the inside of a toilet bowl. Sora's launch--complete with a TikTok style "for you" page--was something of an about-face for Altman, who had previously described social media feeds as "an example of misaligned AI," whose algorithms "are incredible at getting you to keep scrolling." Altman was quick to distance OpenAI from suggestions that it had caved to the temptation to create what he called an AI-powered "slop feed." He wrote: "The team has put great care and thought into trying to figure out how to make a delightful product that doesn't fall into that trap, and has come up with a number of promising ideas."
Exact expressions for double descent and implicit regularization via surrogate random design
It has been shown that models such as deep neural networks, as well as certain so-called interpolating kernels and decision trees, can generalize well in this regime.