Review for NeurIPS paper: Parabolic Approximation Line Search for DNNs

There was a ton of discussion about this paper between reviewers and area chairs, multiple reviewers improved their view of the paper based on the author response, and I read through the paper in detail myself. I was still conflicted after reading it, but I am leaning towards recommending acceptance. However, I *implore* the authors to carefully consider the issues brought up by R2 and R3 as well as the issues that I bring up below. I believe that every single one of the issues brought up can be fixed, and will be extremely-disappointed if these issues are not addressed in the final version (it would make me regret recommending acceptance, and probably be more-harsh on empirical papers in the future, especially by the same authors). Some specific comments from my read-through of the paper: - I agree with the authors that optimization is a mix of theory and empirical work, and it is completely ok for works to be purely empirical if the experiments are done well.