Heated-Up Softmax Embedding

Leveraging these insights, we propose a "heating-up" strategy to train a classifier To overcome the sampling issue, a variety of hard mining strategies (Schroff et al., 2015; Mishchuk In this paper, we show that the temperature parameter in the softmax function, defined by Hinton et al. (2015) for knowledge transfer, plays an important role in determining the distribution of the Compared to the state-of-the-art methods in deep metric learning, the proposed "heating-up" method Siamese networks with contrastive loss (Chopra et al., 2005) was one of the earliest attempts to solve A reasonable solution to address the sampling issue is mining samples that are the most informative for training, also known as "hard mining". Semihard mining (Schroff et al., 2015) tries to find triplets in a training batch, for which the distance of Lifted structured loss (Song et al., 2016) exploits all Proxy NCA (Movshovitz-Attias et al., 2017) proposes to learn semantic proxies for training data and Applying hard mining with proxies is more efficient than with samples. In face verification, quite a few works have shown that training a classifier and using the output of the second last layer as embedding performs reasonably well (Wang et al., 2017b). This paper shows that the scalar can be seen as the temperature parameter of the softmax function in Hinton et al. (2015). The proposed "heating-up" idea is based on an observation that different We define 2 types of training samples as in Figure 1.