5 Tips to Reduce Over and Underfitting Of Forecast Models

Splitting your data and having a Hold-Out process is undoubtedly the simplest model evaluation technique. We take our dataset and split it into two parts: A training set and a test set. After we generate a prediction from the training set, we test a model based on test set data that it has never been exposed to before to see if we get similar results. The typical procedure for this test is to set aside 10% to 30% of randomly chosen data or most recent data and leave it untouched until a model is built and ready to be deployed. Be careful though because if you keep tweaking your model based on the same hold out data then you may be lulled into using the test data to train your model and overfitting the model there without realizing it.