Training Deep Learning models with small datasets

Miguel Romero BSc 1, Yannet Interian PhD 1, Timothy Solberg PhD 2, and Gilmer Valdes PhD 2 1 Master of Science in Data Science, University of San Francisco, San Francisco, CA 2 Department of Radiation Oncology, University of California San Francisco, San Francisco, CA December 17, 2019 Abstract The growing use of Machine Learning has produced significant advances in many fields. For image-based tasks, however, the use of deep learning remains challenging in small datasets. In this article, we review, evaluate and compare current state of the art techniques in training neural networks to elucidate which techniques work best for small datasets. We further propose a path forward for the improvement of model accuracy in medical imaging applications. We observed best results from: one cycle training, discriminative learning rates with gradual freezing and parameter modification after transfer learning. We also established that when datasets are small, transfer learning plays an important role beyond parameter initialization by reusing previously learned features. Surprisingly we observed that there is little advantage in using pre-trained networks in images from another part of the body compared to Imagenet. On the contrary, if images from the same part of the body are available then transfer learning can produce a significant improvement in performance with as little as 50 images in the training data. 1 Introduction The use of machine learning in medical imaging, radiation theranostics and medical physics applications has created tremendous opportunity with research that encompasses: quality assurance [1, 2, 3, 4, 5, 6], outcome prediction [7, 8, 9, 10, 11, 12, 13], segmentation [14, 15, 16, 17] or dosimetric prediction Equal contribution authors. Partially supported by the wicklow AI and medical research initiative at the Data institute.