Fighting Overfitting in Deep Learning

While training the model, we want to get the best possible result according to the chosen metric. And at the same time we want to keep a similar result on the new data. The cruel truth is that we can't get 100% accuracy. And even if we did, the result is still not without errors. There are simply too few test situations to find them. You may ask, what is the matter?