The way we train AI is fundamentally flawed – MIT Technology Review

It's no secret that machine-learning models tuned and tweaked to near-perfect performance in the lab often fail in real settings. This is typically put down to a mismatch between the data the AI was trained and tested on and the data it encounters in the world, a problem known as data shift. For example, an AI trained to spot signs of disease in high-quality medical images will struggle with blurry or cropped images captured by a cheap camera in a busy clinic. Now a group of 40 researchers across seven different teams at Google have identified another major cause for the common failure of machine-learning models. Called "underspecification," it could be an even bigger problem than data shift.