Hyperparameter Optimization -- Intro and Implementation of Grid Search, Random Search and Bayesian Optimization

Usually the first solution that comes to mind when trying to improve a machine learning model is to just add more training data. Additional data usually helps (barring certain situations) but generating high-quality data can be quite expensive. Hyperparameter optimization can save us time and resources by getting the best model performance using the existing data. Hyperparameter optimization, as the name suggests, is the process of identifying the best combination of hyperparameters for a machine learning model to satisfy an optimization function (i.e. In other words, each model comes with multiple knobs and levers that we can change, until we get to the optimized combination.