Reviews: On the Calibration of Multiclass Classification with Rejection

Since 2017, there has been a considerable effort in improving confidence modeling with classifiers, with 2 majors goals: rejection when uncertain, and detecting out-of-distribution examples. In a work that has been mostly empirical and focused on DNNs, this line of work stands out by being mostly theoretical, taking its seeds from work with boosting with abstention. There seems two main contributions in this work, using excellent theoretical derivations. However, their significance may be limited as the authors do not make any effort to connect them to the deep learning literature: 1) Negative result: In some multiclass setting using rejection, it is pointless to train a separate rejector. Solutions that converges towards the Bayes optimal solution requires the rejector to be a function of the Bayes-optimal scoring function, that is it should not be trained separately.