Until a few years ago Artificial Intelligence seemed like a thing from sci-fi movies. The whole concept seemed like fiction or a far fetched dream fed by wishful thinking. Then came personal assistants like Siri, Google Assistant, Bixby, Alexa and Cortana, which made the people realise that they could have something like a Jarvis in their homes as well. However, these are just known as weak AIs. Strong AIs are theoretically able to work with human cognitive abilities.

For those of you unfamiliar, Andrew Ng runs a weekly newsletter where he shares thoughts and new developments in deep learning. I was very interested in something he said in today's newsletter (which can be read here), in which he talks about how deep learning systems still fail in many real scenarios because they are not yet robust to changes in data quality/distributions One of the challenges of robustness is that it is hard to study systematically. How do we benchmark how well an algorithm trained on one distribution performs on a different distribution? Performance on brand-new data seems to involve a huge component of luck. That's why the amount of academic work on robustness is significantly smaller than its practical importance.