Intro to Distributed Deep Learning Systems

Generally speaking, distributed machine learning (DML) is an interdisciplinary domain that involves almost every corner of computer science -- theoretical areas (such as statistics, learning theory, and optimization), algorithms, core machine learning (deep learning, graphical models, kernel methods, etc), and even distributed and storage systems. There are countless problems to be explored and studied in each of these sub-domains. On the other hand, DML is also the most widely adopted and deployed ML technology in industrial production because of its faculty with Big Data. It's easiest to understand DML if you break it into four classes of research problems. Please note, however, that these classes are absolutely not mutually exclusive.