"NIPS Neural Information Processing Systems 8-11th December 2014, Montreal, Canada",,, "Paper ID:","132" "Title:","Fundamental Limits of Online and Distributed Algorithms for Statistical Learning and Estimation" Current Reviews First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The paper considers the effect of memory constraints on some classes of online and distributed algorithms. The main goal of the authors is to show that there exist learning problems where imposing a memory constraint provably hurts the performance of any algorithm, in the sense that the best achievable performance guarantees are worse than some known upper bound on an algorithm that operates without memory constraints. The authors also provide a lower bound on the regret of online algorithms that operate under a specific feedback constraint. The results of the paper follow from an elegant information-theoretic argument concerning hide-and-seek problems where a learner has to detect a biased coordinate of i.i.d.

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

By transferring both features and gradients between different layers, shortcut connections explored by ResNets allow us to effectively train very deep neural networks up to hundreds of layers. However, the additional computation costs induced by those shortcuts are often overlooked.

Nesterov acceleration to decrease the required number of gradient steps.

Neurons mainly communicate through spikes, and much effort has been spent to understand how the dynamics of spiking neural networks (SNNs) relates to their connectivity.

RDP accounting routines implemented in open-source libraries (e.g.

While highly accurate, the resulting predictors are large, slow, and opaque as compared to their constituents.