Keras Horovod Distributed Deep Learning on Steroids

Keras is definitely the weapon of choice when it comes to building deep learning models ( with tensorflow backend). At SearchInk, we are solving varied problems in the field of document analysis by architecting and implementing deep learning models. One of the bigger challenges in doing this is the time taken to run each experiment. With the need for more and more experimentation to be carried out in shorter spans of time, we decided it was the right time for us to start distributed computations on the GPU for deep learning models. We were evaluating different options on how to perform distributed GPU computing and we stumbled upon Horovod.