How To Be Confident In Your Neural Network Confidence

Those notes are based on the research paper " On Calibration of Modern Neural Networks" by (Guo et al, 2017.). Very large and deep models, as ResNet, are far more accurate than their older counterparts, as LeNet, on computer vision datasets such as CIFAR100. However while they are better at classifying images, we are less confident in their own confidence! Most neural networks for classification uses as last activation a softmax: it produces a distribution of probabilities for each target (cat, dog, boat, etc.). We may expect that if for a given image, our model associate a score of 0.8 to the target'boat', our model is confident at 80% that this is the right target.