Distributed Deep Learning -- Illustrated

In this article, I will illustrate how distributed deep learning works. I have created animations that should help you get a high-level understanding of distributed deep learning. But let's start with the basics. Graphics processing units (GPUs) are specialized cores that can perform multiple, simultaneous mathematical computations. Deep learning computations can be broken down into a series of matrix multiplications and that is where GPUs excel over CPUs.