DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier

DeGAN: Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier Sravanti Addepalli, Gaurav Kumar Nayak, Anirban Chakraborty, R. V enkatesh Babu Department of Computational and Data Sciences Indian Institute of Science, Bangalore, India {sravantia, gauravnayak, anirban, venky } @iisc.ac.in Abstract In this era of digital information explosion, an abundance of data from numerous modalities is being generated as well as archived everyday. However, most problems associated with training Deep Neural Networks still revolve around lack of data that is rich enough for a given task. Data is required not only for training an initial model, but also for future learning tasks such as Model Compression and Incremental Learning. A diverse dataset may be used for training an initial model, but it may not be feasible to store it throughout the product life cycle due to data privacy issues or memory constraints. We propose to bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a given trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples from a trained classifier, using a novel Data-enriching GAN (DeGAN) framework. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance for the tasks of Data-free Knowledge Distillation and Incremental Learning on benchmark datasets. We further demonstrate that our proposed framework can enrich any data, even from unrelated domains, to make it more useful for the future learning tasks of a given network. 1 Introduction The performance and generalizability of Deep Neural Networks largely depend on the amount and quality of training data available. Several successful implementations of tasks such as classification, object detection and segmentation leverage very large, class-balanced and diverse datasets. In addition to training an initial network, data is also required for future updates to the model. This makes it important for training data to be available throughout the life cycle of a product. While data collection is a challenge in itself, storing the data for future use could also be a concern due to data confidentiality constraints, privacy issues, or memory costs.