\epsilon -Softmax: Approximating One-Hot Vectors for Mitigating Label Noise

Noisy labels pose a common challenge for training accurate deep neural networks. To mitigate label noise, prior studies have proposed various robust loss functions to achieve noise tolerance in the presence of label noise, particularly symmetric losses. However, they usually suffer from the underfitting issue due to the overly strict symmetric condition. In this work, we propose a simple yet effective approach for relaxing the symmetric condition, namely ** \epsilon -softmax**, which simply modifies the outputs of the softmax layer to approximate one-hot vectors with a controllable error \epsilon . Essentially, *** \epsilon -softmax** not only acts as an alternative for the softmax layer, but also implicitly plays the crucial role in modifying the loss function.*