Fine-tune your Classifier: Finding Correlations With Temperature

Nevertheless, such Temperature is a widely used hyperparameter in various tasks strategies for determining a good temperature may be suboptimal involving neural networks, such as classification or metric or computationally too cumbersome. Surprisingly, there learning, whose choice can have a direct impact on the model are very few studies proposing strategies for determining an performance. Most of existing works select its value using optimal temperature. In this paper, we focus on the particular hyperparameter optimization methods requiring several runs problem that, given a classification task, we need to find a to find the optimal value. We propose to analyze the impact of correlation between an optimal value for the temperature and temperature on classification tasks by describing a dataset as a statistics describing the dataset such as complexity, dimension, set of statistics computed on representations on which we can number of classes, etc. build a heuristic giving us a default value of temperature. We study the correlation between these extracted statistics and the observed optimal temperatures.