Deep Learning
Fast Second Order Stochastic Backpropagation for Variational Inference
Kai Fan, Ziteng Wang, Jeff Beck, James Kwok, Katherine A. Heller
We propose a second-order (Hessian or Hessian-free) based optimization method for variational inference inspired by Gaussian backpropagation, and argue that quasi-Newton optimization can be developed as well. This is accomplished by generalizing the gradient computation in stochastic backpropagation via a reparametrization trick with lower complexity. As an illustrative example, we apply this approach to the problems of Bayesian logistic regression and variational auto-encoder (V AE). Additionally, we compute bounds on the estimator variance of intractable expectations for the family of Lipschitz continuous function. Our method is practical, scalable and model free. We demonstrate our method on several real-world datasets and provide comparisons with other stochastic gradient methods to show substantial enhancement in convergence rates.
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question
Haoyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, Wei Xu
Neural Information Processing Systems http://nips.cc/
Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming
This precludes applications that involve verification-agnostic networks, i.e., networks not specially trained for verification.
Extending Stein's unbiased risk estimator to train deep denoisers with correlated pairs of noisy images
Magauiya Zhussip, Shakarim Soltanayev, Se Young Chun
Neural Information Processing Systems http://nips.cc/
Counterfactual Vision and Language Navigation: Unravelling the Unseen
We provide a brief background on counterfactuals. Further details can be found in [1].
Convolutional Networks on Graphs for Learning Molecular Fingerprints
We introduce a convolutional neural network that operates directly on graphs. These networks allow end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. The architecture we present generalizes standard molecular feature extraction methods based on circular fingerprints. We show that these data-driven features are more interpretable, and have better predictive performance on a variety of tasks.
Towards modular and programmable architecture search
However, current systems focus on specific use-cases (e.g.
Our programmable and modular language supports architecture search
We will add additional discussion to the appendix.
Multiscale Deep Equilibrium Models
Is implicit deep learning relevant for general pattern recognition tasks?
Neural Path Features and Neural Path Kernel: Understanding the role of gates in deep learning Chandrashekar Lakshminarayanan and Amit Vikram Singh
A deep neural network (DNN) with ReLU activations has many gates, and the on/off status of each gate changes across input examples as well as network weights. For a given input example, only a subset of gates are active, i.e., on, and the sub-network of weights connected to these active gates is responsible for producing