Appendix A Broader Impact

Overconfidence in deep neural networks could easily lead to deployments where predictions are made that should have been withheld. For validation set, on the other hand, we care about the confidence of the "top predicted class". Independent binning: when training samples and validation samples are grouped independently into their respective training-bins and validation-bins (Figure 1). The binning is adaptive with 15 equal-mass bins. Figure 10: Common binning: training samples are grouped using the bin boundaries of the validation-bins.