Review for NeurIPS paper: Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate

Neural Information Processing Systems 

Thank you for submitting your work to NeurIPS. All four reviewers were enthusiastic about the paper, and I am happy to accept it. In the final revision, please address reviewers' feedback. Especially, please make sure to address the reviewers' 2 remark "authors argue that their results indicate that large learning rates do not generalize well, but a better presentation would be to say that they show that large effective learning rates generalize well.". Indeed, it is somewhat a strawman argument to say that other researchers claim that small LR never generalize.