Why Warmup the Learning Rate? Underlying Mechanisms and Improvements

Open in new window