Developmental Pretraining (DPT) for Image Classification Networks

The advent of Deep Learning (DL) has massively aided the Artificial Intelligence community, especially in the realm of object recognition. One of the critical reasons for the success of DL has been the availability of massive image datasets [1] and the computational power offered by modern Graphics Processing Units (GPUs) that are able to accommodate the large amounts of data required by Deep Networks. State-of-the-art image recognition networks like the ResNet family [2], VGG networks [3], EfficientNet models [4] and the recently introduced Vision Transformers [5] require extremely large amounts of data compared to their classical Machine Learning (ML) counterparts [6]. This characteristic requirement for large amounts of data becomes a problem in fields where data availability is low like in medical fields [7]. A common approach to this problem is Transfer Learning [8] which consists of pre-training a network on a large dataset like ImageNet [1] and fine tune the network on a smaller dataset that is relevant to the recognition problem at hand.