Health Checks for Machine Learning - A Guide to Model Retraining and Evaluation

In 2013, IBM and University of Texas Anderson Cancer Center developed an AI based Oncology Expert Advisor. According to IBM Watson, it analyzes patients medical records, summarizes and extracts information from vast medical literature, research to provide an assistive solution to Oncologists, thereby helping them make better decisions. According to an article on The Verge, the product demonstrated a series of poor recommendations. Like recommending a drug to a lady suffering from bleeding that would increase the bleeding. "A parrot with an internet connection" - were the words used to describe a modern AI based chat bot built by engineers at Microsoft in March 2016. 'Tay', a conversational twitter bot was designed to have'playful' conversations with users. It was supposed to learn from the conversations. It took literally 24 hours for twitter users to corrupt it.