Grouped Sequential Optimization Strategy -- the Application of Hyperparameter Importance Assessment in Deep Learning

In recent years, the rapid advancement of deep learning has led to significant breakthroughs across a wide range of applications, from computer vision to natural language processing, where hyperparameter optimization (HPO) has become increasingly vital in constructing models that achieve optimal performance. As the demand for HPO has been growing, the computational and time costs associated with it have become a significant bottleneck [1]. In this context, Hyperparameter Importance Assessment (HIA) has emerged as a promising solution. By evaluating the importance weights of individual hyperparameters and their combinations within specific models, HIA provides valuable insights into which hyperparameters most significantly impact model performance [2]. With this understanding, deep learning practitioners can focus on optimizing only those hyperparameters that have a more pronounced effect on performance. For less critical hyperparameters, users can reduce the search space during optimization or even fix them at certain values, thereby saving time in the model optimization process [3]. Although there has been considerable exploration of HIA, most existing studies have primarily focused on introducing new HIA methods or determining the importance rankings of hyperparameters for specific models within certain application scenarios. However, there has been limited exploration of how these insights can be strategically applied to enhance the efficiency of the optimization process. To address the challenges in the current research landscape, this paper aims to use Convolutional Neural Networks (CNNs) as the research case to introduce HIA into the deep learning pipeline, demonstrating that the insights gained from HIA can effectively enhance the efficiency of hyper-Second Conference on Parsimony and Learning (CPAL 2025).