Overconfidence in deep neural networks could easily lead to deployments where predictions are made that should have been withheld. Figure 7: ResNet-50 trained onCIFAR-10 using focal lossγ = 0,3,4,5. Similarly, the confidence of the top predicted classˆy (for the training sample) isdenoted byˆptrain,top and theaverage equivalent inabinbyCtrain,top. Forthe training set, we care only about the confidence ofthe "true class"ˆptrain,true asthat isthe quantity which gets manipulated by some loss function. For validation set, on the other hand, we care about the confidence of the "top predicted class".

This success stems from focal loss regularizing the entropyofthe model'sprediction (controlled by the parameterγ), thereby reining in the model's overconfidence.