Transfer Learning: A Shortcut for Training Deep Learning Models

When to use Transfer Learning? In this approach, the last few fully connected layers of the pre-trained model are removed and replaced with a shallow neural network. The layers of the pre-trained model are frozen, and only the shallow neural network is trained with the available target dataset. The features extracted by the pre-trained model help the shallow to learn and perform well on the target task. The benefit of this approach is the low chance of overfitting, as we are only training the last few layers of the model, keeping the initial layers fixed.