Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift

We might hope that when faced with unexpected inputs, well-designed software systems would fire off warnings. Machine learning (ML) systems, however, which depend strongly on properties of their inputs (e.g. the i.i.d.