Deep metric learning has attracted much attention in recent years, due to seamlessly combining the distance metric learning and deep neural network. Many endeavors are devoted to design different pair-based angular loss functions, which decouple the magnitude and direction information for embedding vectors and ensure the training and testing measure consistency. However, these traditional angular losses cannot guarantee that all the sample embeddings are on the surface of the same hypersphere during the training stage, which would result in unstable gradient in batch optimization and may influence the quick convergence of the embedding learning. In this paper, we first investigate the effect of the embedding norm for deep metric learning with angular distance, and then propose a spherical embedding constraint (SEC) to regularize the distribution of the norms. SEC adaptively adjusts the embeddings to fall on the same hypersphere and performs more balanced direction update. Extensive experiments on deep metric learning, face recognition, and contrastive self-supervised learning show that the SEC-based angular space learning strategy significantly improves the performance of the state-of-the-art.

Weaknesses: 1) I think the main drawback of this paper is that, the design of the SEC loss is somehow too straightforward and trivial. It simply minimizes the distance between the norm of each feature and the average norm, which acts as an additional term of the existing losses. A more elegant design should be expected for the NeurIPS level. It seems the latter, but in this case how to update \mu during training? Whenever the parameters of the metric are updated, all the features are changed, and \mu should be re-calculated.

This paper points out a widespread problem with angular losses, and proposes a simple, elegant scheme to address the problem (regularizing each embedding to lie on a shell), getting moderate but consistent improvements across a range of problem settings and datasets. As pointed out by Reviewer 5, the majority of the theoretical results were already known in Section 3.3 of "Heated-Up Softmax Embedding" (2018, unpublished, https://arxiv.org/abs/1809.04157). That paper, however, did not really propose a solution to the problem, merely noted its existence. Reviewer 5 also complains that the interaction with the Adam optimizer is under-explored in this work. "Improved Deep Metric Learning with Multi-class N-pair Loss Objective," also regularized the L2 norm of embedding vectors (towards 0; see their Section 3.2.2).

As recruitment and talent acquisition have become more and more competitive, recruitment firms have become more sophisticated in using machine learning (ML) methodologies for optimizing their day to day activities. But, most of published ML based methodologies in this area have been limited to the tasks like candidate matching, job to skill matching, job classification and normalization. In this work, we discuss a novel task in the recruitment domain, namely, application count forecasting, motivation of which comes from designing of effective outreach activities to attract qualified applicants. We show that existing auto-regressive based time series forecasting methods perform poorly for this task. Henceforth, we propose a multimodal LM-based model which fuses job-posting metadata of various modalities through a simple encoder. Experiments from large real-life datasets from CareerBuilder LLC show the effectiveness of the proposed method over existing state-of-the-art methods.

Deep metric learning has attracted much attention in recent years, due to seamlessly combining the distance metric learning and deep neural network. Many endeavors are devoted to design different pair-based angular loss functions, which decouple the magnitude and direction information for embedding vectors and ensure the training and testing measure consistency. However, these traditional angular losses cannot guarantee that all the sample embeddings are on the surface of the same hypersphere during the training stage, which would result in unstable gradient in batch optimization and may influence the quick convergence of the embedding learning. In this paper, we first investigate the effect of the embedding norm for deep metric learning with angular distance, and then propose a spherical embedding constraint (SEC) to regularize the distribution of the norms. SEC adaptively adjusts the embeddings to fall on the same hypersphere and performs more balanced direction update.