Common Mistakes in Hyper-Parameters Tuning

Although the principle is straightforward, this method is still error-prone. Here is a list of the most common mistakes I have encountered. This error I've seen it happen quite a few times. Students define a grid on a parameter, run GridSearchCV, extract the hyper-parameter value corresponding to the best score, and …. that's it! Depending on how well the grid was defined, just looking at the best score and its corresponding hyper-parameter value might not be enough to draw the right conclusions.