Facebook is speeding up training for visual recognition models

Every minute spent training a deep learning model is a minute not doing something else and in today's fast paced world of research, that minute is worth a lot. Facebook published a paper this morning detailing its personal approach to this problem. The company says its managed to reduce the training time of a ResNet-50 deep learning model on ImageNet from 29 hours to one. Facebook managed to reduce training time so dramatically by distributing training in larger "minibatches" across a greater number of GPUs. In the previous benchmark case, batches of 256 images were spread across eight GPUs.