Reviews: Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with Dirichlet calibration

Neural Information Processing Systems 

POST REBUTTAL COMMENTS This did not affect my decision and score, but the citations on calibration should be improved. The oldest work on calibration that the authors cite is from 2000 (Platt). The definitions of calibration/calibration error, and many of the key ideas were proposed several decades ago by statisticians and meteorologists like Brier, Murphy, Winkler, Deegroot, Fienberg. The calibration error metric proposed there is different from ECE (root-mean-squared instead of mean absolute value), it would be good if the authors can mention that the RMS calibration error is another possible metric. I've included some pointers to the literature below (there are many other papers, but it should be OK to just cite a few): - Verification of forecasts expressed in terms of probability.