Imbalanced Data? Stop Using ROC-AUC and Use AUPRC Instead

The Receiver Operating Characteristic -- Area Under the Curve (ROC-AUC) measure is widely used to assess the performance of binary classifiers. However, sometimes, it is more appropriate to evaluate your classifier based on measuring the Area Under the Precision-Recall Curve (AUPRC). We will present a detailed comparison between these two measures, accompanied by empirical results and graphical illustrations. Scikit-learn experiments are also available in a corresponding notebook. I'll assume you're familiar with precision and recall and the elements of the confusion matrix (TP, FN, FP, TN).