Review for NeurIPS paper: On the training dynamics of deep networks with L_2 regularization
–Neural Information Processing Systems
Weaknesses: ((1)) If I could have access to the test set, then why bother tuning l2 regularisation to get optimal on the test set? Technically, I could run a brute-force algorithm to find an optimal set of parameters without tuning any other hyperparameters. I do think the submission violates the ethics of machine learning research. I understand that theoretical work generally considers the generalisation gap between the training set and the test set, however, the submission is an empirical work on hyperparameter tuning for optimal l2 regularisation that gives the highest test set accuracy. Therefore, a validation set is required for tuning, and then it should be tested on the test set afterward.
Neural Information Processing Systems
Jan-23-2025, 06:43:46 GMT
- Technology: