Ground Metric Learning

We consider in this paper the problem of supervised metric learning on normalized histograms. Normalized histograms arise frequently in natural language processing, computer vision, bioinformatics and more generally areas involving complex datatypes. Objects of interest in such areas are usually simplified and each represented as a bag of smaller features. The occurrence frequencies of each of these features in the considered object can then be represented as a histogram. For instance, the representation of images as histograms of pixel colors, SIFT or GIST features (Lowe 1 1999, Oliva and Torralba 2001, Douze et al. 2009); texts as bags-of-words or topic allocations (Joachims 2002, Blei et al. 2003, Blei and Lafferty 2009); sequences as n-grams counts (Leslie et al. 2002) and graphs as histograms of subgraphs (Kashima et al. 2003) all follow this principle. Various distances have been proposed in the statistics and machine learning literatures to compare two histograms(Deza and Deza 2009,§14), (Rachev 1991). Our focus is in this paper is on the family of transportation distances, which is both well motivated theoretically (Villani 2003, §7), (Rachev 1991, §5) and works well empirically (Rubner et al. 1997; 2000, Pele and Werman 2009). Transportation distances are particularly popular in computer vision, where, after the influential work of Rubner et al. (1997), they were called Earth Mover's Distances (EMD). Transportation distances in machine learning can be thought of as metadistances that build upon a metric on the features to form a distance on histograms of features.