Validation Set Over-fitting because of Random Initialization & Selection Bias

Deep Learning models are called the universal function approximators. Their strength comes from their great ability of modeling the relationship of given input and output. However, this is also their primary weakness for coming up with generalizable solutions to a problem and what makes them so prone to over-fitting (memorizing) the training-set and not working with new data. The current method of ensuring the generalization of Deep Learning models was simply using a validation set to decide on how many iterations (epochs) should be done for training a model; or in other words, early stopping. Then, the Data Scientist would test the trained model on a blind test-set to ensure sure that none of the training hyper-parameters are also over-fitting.