A Priori Generalizability Estimate for a CNN

A Priori Generalizability Estimate for a CNN Cito Balsells 1,2 Beatrice Riviere 1 David Fuentes 2 1 Department of Computational Applied Mathematics and Operations Research, The Ken Kennedy Institute, Rice University, 6100 Main St, Houston, TX, 77005, USA 2 Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA Abstract We formulate truncated singular value decompositions of entire convolutional neural networks. We demonstrate the computed left and right singular vectors are useful in identifying which images the convolutional neural network is likely to perform poorly on. To create this diagnostic tool, we define two metrics: the Right Projection Ratio and the Left Projection Ratio. The Right (Left) Projection Ratio evaluates the fidelity of the projection of an image (label) onto the computed right (left) singular vectors. We observe that both ratios are able to identify the presence of class imbalance for an image classification problem. Additionally, the Right Projection Ratio, which only requires unlabeled data, is found to be correlated to the model's performance when applied to image segmentation. This suggests the Right Projection Ratio could be a useful metric to estimate how likely the model is to perform well on a sample.