Scaling TensorFlow and Caffe to 256 GPUs - IBM Systems Blog: In the Making

Deep learning has taken the world by storm in the last four years, powering hundreds of consumer web and mobile applications that we use every day. But the extremely long training times in most frameworks present a hurdle that's curtailing the broader proliferation of deep learning. It currently may take days or even weeks to train large AI models with big data sets to get the right accuracy levels. At the crux of this problem is a technical limitation. The popular open-source deep-learning frameworks do not seem to run as efficiently across multiple servers.