Cyclical Learning Rates with Keras and Deep Learning - PyImageSearch

In this tutorial, you will learn how to use Cyclical Learning Rates (CLR) and Keras to train your own neural networks. Using Cyclical Learning Rates you can dramatically reduce the number of experiments required to tune and find an optimal learning rate for your model. Last week we discussed the concept of learning rate schedules and how we can decay and decrease our learning rate over time according to a set function (i.e., linear, polynomial, or step decrease). Cyclical Learning Rates take a different approach. In practice, using Cyclical Learning Rates leads to faster convergence and with fewer experiments/hyperparameter updates.