Explainable AI or Halting Faulty Models ahead of Disaster

Experienced machine learning experts will know about the challenge's complexity and rightfully question the results' validity. At the same time, submissions like this Notebook illustrate how the Titanic competition's leaderboard can be forged effortlessly; A top-performing model can be created by collecting and including the publicly accessible list of survivors. Clearly, such overfit models only work for one very specific use case and are virtually useless for predicting outcomes in any other situation (not to mention the ethics of cheating). So, how can we make sure we have trained or are provided with a model that we can actually use in production? How can machine learning systems be deployed without likely ensuing disaster?