Exploring Weight Balancing on Long-Tailed Recognition Problem

Datasets with an equal number of samples per class, such as MNIST [Lecun et al., 1998], CIFAR10, and CIFAR100 [Krizhevsky, 2009], are often used, when we evaluate classification models and training methods in machine learning. However, it is empirically known that the size distribution in the real world often shows a type of exponential distribution called Pareto distribution [Reed, 2001], and the same is true for the number of per-class samples in classification problems [Li et al., 2017, Spain and Perona, 2007]. Such distributions are called long-tailed data due to the shape of the distribution since some classes (head classes) are often sampled and many others (tail classes) are not sampled very often. Long-tailed recognition (LTR) is used to attempt to improve the accuracy of classification models on uniform distribution when training data shows such a distribution. There is a problem in LTR that the head classes have large sample size; thus, the output is biased toward them. This reduces the overall and tail class accuracy because tail classes make up the majority [Zhang et al., 2021]. Various methods have been developed for LTR, such as class-balanced loss (CB) [Cui et al., 2019], augmenting samples of tail classes [Wang et al., 2021], two-stage learning [Kang et al.,