In the reminder of the paper we will try whenever possible to formally introduce functions by clearly stating domain and co-domain.

Neural Information Processing Systems http://nips.cc/

There has been an increased interest in discovering heuristics for combinatorial problems on graphs through machine learning. While existing techniques have primarily focused onobtaining high-quality solutions, scalability tobillion-sized graphs has not been adequately addressed.

Prior work tries to improve the sampling locality by enforcing all the training jobs loading the same dataset in the same order and pace. However, such a solution isonly efficient under strong constraints: alljobs are trained onthe same dataset with the same starting moment and training speed. In this paper, we propose a new data loading method for efficiently training parallel DNNs with much flexible constraints. Our method is still highly efficient when different training jobs use different but overlapped datasets and have different starting moments andtrainingspeeds.