dynamic hyper-parameter
[P][D] Dynamic Hyper-parameters
They are set before training starts, either by intuition or a hyper-parameter search. They either stay static or change based on a pre-determined schedule. We are introducing dynamic hyper-parameters which can be manually adjusted during the training based on model training stats. Hyper-parameters are parameters that control the learning process of models, such as the learning rate, batch size, and weight decay. The model might not learn if the hyper-parameters are not set correctly.