What Is Deep Transfer Learning and Why Is It Becoming So Popular?

As we already know, large and effective deep learning models are data-hungry. They require training with thousands or even millions of data points before making a plausible prediction. Training is very expensive, both in time and resources. For example, the popular language representation model BERT, developed by Google, has been trained on 16 Cloud TPUs (64 TPU chips total) for 4 days. Put in perspective, this is 60 desktop computers running non-stop for 4 days.