A Solution to the Memory Limit Challenge in Big Data Machine Learning

The model training process in big data machine learning is both computation- and memory-intensive. Many parallel machine learning algorithms consist of iterating a computation over a training dataset and updating the related model parameters until the model converges. In the Big Data era, both the volume of a dataset and the number of model parameters can be huge. To accelerate the performance of the iterative computation, it's common to cache the training data and model parameters into memory. However, due to the limitations of memory, in many scenarios, it might not all fit.