Deep Divergence Learning

These methods, known as Mahalanobis metric learning approaches, have been analyzed Classical linear metric learning methods have recently theoretically, are scalable, and usually involve convex optimization been extended along two distinct lines: problems that can be solved globally (Kulis, 2013; deep metric learning methods for learning embeddings Bellet et al., 2015). of the data using neural networks, and Classical metric learning methods have been extended along Bregman divergence learning approaches for extending various axes; two important directions are deep metric learning learning Euclidean distances to more general and Bregman divergence learning. Deep metric learning divergence measures such as divergences over approaches replace the linear mapping learned in Mahalanobis distributions. In this paper, we introduce deep metric learning methods with more general mappings Bregman divergences, which are based on learning that are learned via neural networks (Hoffer & Ailon, and parameterizing functional Bregman divergences 2015; Chopra et al., 2005). On the other hand, Bregman using neural networks, and which unify divergence methods replace the squared Euclidean distance and extend these existing lines of work. We show with arbitrary Bregman divergences (Bregman, 1967), and in particular how deep metric learning formulations, learn the underlying generating function of the Bregman kernel metric learning, Mahalanobis metric divergence via piecewise linear approximators (Siahkamari learning, and moment-matching functions for et al., 2019) or convex combinations of existing basis functions comparing distributions arise as special cases of (Wu et al., 2009).