Deep Learning Epic Fail – Right Answer - Wrong Reason

Summary: Got a good AUC on your hold out data? Think that proves that it's safe to put the model into production. This article shows you some of the pitfalls in this new era of black box Deep Learning Neural Nets and a method for identifying potentially devastating errors. Recently we've been reading that the increased adoption of Deep Learning (DL) image, text, and voice processing tools has been driven by the fact that major developers like Google and Facebook have been able to achieve accuracies of 99% or greater. Deep Learning is indeed on a roll. But before you rush out to add DL to your app, take a step back and consider whether the answers you're getting are in fact right even with very high AUC scores. Deep Learning is an evolution of Neural Nets (NN) and NNs have been around for a long time.