Reviews: Addressing Failure Prediction by Learning Model Confidence

Originality: The use of the True Class Probability (TCP) is novel in the area of uncertainty estimation for the task of detecting misclassifications. However, as the TCP cannot be known at test time, the authors use an additional network component to predict this score based intermediate features derived from the predictive model. This is very similar to the task of confidence score estimation in speech recognition (https://arxiv.org/abs/1810.13024, While the use of the TCP as a target is novel, confidence score estimation is not, thus the work has limited novelty. Quality: The paper is technically sound, experiments are sensible and in line with standard practice in the area. The authors are generally honest about the evaluation of their work, however, they do not analyse the limitations of the given approach.