How to Automate Hyperparameter Optimization - DZone AI

In the machine learning and deep learning paradigm, model "parameters" and "hyperparameters" are two frequently used terms where "parameters" define configuration variables that are internal to the model and whose values can be estimated from the training data and "hyperparameters" define configuration variables that are external to the model and whose values cannot be estimated from the training data (What is the Difference Between a Parameter and a Hyperparameter? Thus, the hyperparameter values need to be manually assigned by the practitioner. Every machine learning and deep learning model that we make has a different set of hyperparameter values that need to be fine-tuned to be able to obtain a satisfactory result. Compared to machine learning models, deep learning models tend to have a larger number of hyperparameters that need optimizing in order to get the desired predictions due to its architectural complexity over typical machine learning models. Repeatedly experimenting with different value combinations manually to derive the optimal hyperparameter values for each of these hyperparameters can be a very time consuming and tedious task that requires good intuition, a lot of experience, and a deep understanding of the model. Moreover, some hyperparameter values may require continuous values, which will have an undefined number of possibilities, and even if the hyperparameters require a discrete value, the number of possibilities is enormous, thus manually performing this task is rather difficult.