Train your Deep Learning Faster: FreezeOut

The authors of this paper propose a method to increase training speed by freezing layers. The authors demonstrated a way to freeze the layers one by one as soon as possible, resulting in fewer and fewer backward passes, which in turn lowers training time. The authors experimented with different values for Equation 2.1 The authors tried scaling the initial learning rate so that each layer was trained for an equal amount of time. I demonstrated 2(and half of my own) very recent and novel techniques to improve accuracy and lower training time by fine tuning learning rates.