Novel Uncertainty Framework for Deep Learning Ensembles

Deep learning (DL) algorithms have successfully solved real-world classification problems from a variety of fields, including recognizing handwritten digits and identifying the presence of key diagnostic features in medical images [18, 16]. A typical classification challenge for a DL algorithm consists of training the algorithm on an example data set, then using a separate set of test data to evaluate its performance. The aim is to provide answers that are as accurate as possible, as measured by the true positive rate (TPR) and the true negative rate (TNR). Many DL classifiers, particularly those using a softmax function in the very last layer, yield a continuous score, h; A step function is used to map this continuous score to each of the possible categories that are being classified. TPR and TNR scores are then generated for each separate variable that is being predicted by setting a threshold parameter that is applied when mapping h to the decision. Values above this threshold are mapped to positive predictions, while values below it are mapped to negative predictions. The ROC curve is then generated from these pairs of TPR/TPN scores.