Baidu's 'Ring Allreduce' Library Increases Machine Learning Efficiency Across Many GPU Nodes

Baidu's SVAIL used the ring allreduce algorithm to train state of the art speech recognition models, and now it hopes that others will take advantage of the group's implementation to do even more interesting things. The group released its ring allreduce implementation as both a standalone C library as well as a patch for TensorFlow.