How to optimize storage for AI, machine learning and deep learning

Many researchers have shown that the accuracy of deep learning models improves with larger data sets. The improvement is so significant that researchers often use sophisticated data augmentation techniques to synthetically generate extra data for model training. For example, this data set from ImageNet used to benchmark various deep learning image classification algorithms contains more than 14 million images with a million annotations. While the ResNet-50 model often used to benchmark image classification hardware is just over 100 MB in size. The models, which ideally are held in memory, must be continually fed with data and that often results in the storage system becoming the bottleneck to overall performance.