How to Reduce the Training Time of Your Neural Network from Hours to Minutes

In part 1 of the series we looked at how it is possible to get a 1500x speed-up in IO operations with a few lines of Python using the multiprocessing module. In this article, we will look at parallelising a deep learning code and reducing the training time from roughly 13 hours to 13 minutes! As a data scientist, you will eventually face the following problem (if you haven't faced it already). "I have a neural network to train but the input data doesn't fit in memory!" or "My neural network takes forever to train with this amount of data!" It would surely be a pity to exclude a substantial part of your data for training, or wait for hours (even days) for your neural network to finish training.