Deep Learning
Supplementary Materials for Nimble: Lightweight and Parallel GPU T ask Scheduling for Deep Learning Appendix A Proofs on the Stream Assignment Algorithm of Nimble
In this section, we provide detailed proofs on the theorems presented in Section 4.2. We assume that the computation graph of a neural network is given. Here we define important concepts and terminologies used in the following proofs. F or any (u,v) E, f ( u) = f (v) or there exists a path P E from u to v such that P ฮ null= . Prior to the proof of Theorem 1-2, we describe and prove Lemma 1 and Lemma 2. Lemma 1. We will prove by contradiction.
Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning
Nimble introduces a novel technique called ahead-of-time (AoT) scheduling.
Neural Networks with Cheap Differential Operators
Tian Qi Chen, David K. Duvenaud
Neural Information Processing Systems http://nips.cc/
Shallow RNN: Accurate Time-series Classification on Resource Constrained Devices
Don Dennis, Durmus Alp Emre Acar, Vikram Mandikal, Vinu Sankar Sadasivan, Venkatesh Saligrama, Harsha Vardhan Simhadri, Prateek Jain
Neural Information Processing Systems http://nips.cc/
Theory-Inspired Path-Regularized Differential Network Architecture Search
The methodology of NAS is to automatically search for a directed graph and its edges from a huge search space.
Adversarial Distributional Training for Robust Deep Learning Yinpeng Dong
Moreover, a single attack algorithm could be insufficient to explore the space of possible perturbations.
An efficient nonconvex reformulation of stagewise convex optimization problems
Convex optimization problems with staged structure appear in several contexts, including optimal control, verification of deep neural networks, and isotonic regression. Off-the-shelf solvers can solve these problems but may scale poorly. We develop a nonconvex reformulation designed to exploit this staged structure. Our reformulation has only simple bound constraints, enabling solution via projected gradient methods and their accelerated variants.
