Deep Learning
Author response: 'Stationary Activations for Uncertainty Calibration in Deep Learning ' NeurIPS: # 5154
We thank the anonymous reviewers for their enthusiasm and detailed comments on the manuscript. We start by addressing R2's concerns as they had the lowest score. This link is explicit as shown in this paper. The choice of kernel/activation function is up to the modelling task and'expert We merely provide a building block. The Matérn is a widely used prior, and worth adding to the NN tool set.
Ouroboros: On Accelerating Training of Transformer-Based Language Models
Qian Yang, Zhouyuan Huo, Wenlin Wang, Lawrence Carin
We also prove that our proposed algorithm is guaranteed to converge to critical points for non-convex problems.
On the Fairness of Disentangled Representations
Francesco Locatello, Gabriele Abbati, Thomas Rainforth, Stefan Bauer, Bernhard Schölkopf, Olivier Bachem
Neural Information Processing Systems http://nips.cc/
Trivializations for Gradient-Based Optimization on Manifolds Mario Lezcano-Casado Department of Mathematics University of Oxford Oxford, mario.lezcanocasado@maths.ox.ac.uk
To this end, we prove a formula for the gradient of the exponential of matrices, which can be of practical interest on its own.
Direct Optimization through $\arg \max$ for Discrete Variational Auto-Encoder
Guy Lorberbom, Andreea Gane, Tommi Jaakkola, Tamir Hazan
Neural Information Processing Systems http://nips.cc/
Zero-shot Learning via Simultaneous Generating and Learning
However, deep learning as a non-linear regression tool based on statistics mostly suffers from the insufficient or non-existing training data, which is the usual case and should be overcome for autonomous learning systems.