Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering