Reviews: Using Statistics to Automate Stochastic Optimization

The paper proposes to automate the tuning of learning rate schedules in stochastic gradient methods, which is an important problem. In this regards, the authors propose a statistical test to determine when to decay the learning rate. The statistical test build upon a prior work with simple albeit useful extensions. Resulting statistical test is simple and can be deployed easily. There are some concerns regarding mismatch between theoretical assumptions made and the setup in practice. Nevertheless, empirically the learning rate schedule followed by decaying when the test is true seems to be almost competitive with hand-tuned methods.