Universal Metric Learning with Parameter-Efficient Transfer Learning

A common practice in metric learning is to train and test an embedding model for each dataset. This dataset-specific approach fails to simulate real-world scenarios that involve multiple heterogeneous distributions of data. In this regard, we introduce a novel metric learning paradigm, called Universal Metric Learning (UML), which learns a unified distance metric capable of capturing relations across multiple data distributions. UML presents new challenges, such as imbalanced data distribution and bias towards dominant distributions. To address these challenges, we propose Parameter-efficient Universal Metric leArning (PUMA), which consists of a pre-trained frozen model and two additional modules, stochastic adapter and prompt pool. These modules enable to capture dataset-specific knowledge while avoiding bias towards dominant distributions. Additionally, we compile a new universal metric learning benchmark with a total of 8 different datasets. PUMA outperformed the state-of-the-art dataset-specific models while using about 69 times fewer trainable parameters. Deep metric learning stands out as the prominent method for learning semantic distance metrics. It aims to learn highly nonlinear distance metrics through deep neural networks that approximate the actual underlying semantic similarity between samples. While metric learning methods have achieved remarkable progress, they focus on learning metrics unique to a specific dataset under the assumption that both training and test datasets share a common distribution. However, real-world applications often violate this assumption and involve multiple heterogeneous data distributions. To tackle this issue using conventional methods, it is imperative to train multiple models as shown Figure 1 (a) and subsequently combine them through ensemble techniques or toggle between the models based on the query. Such procedures are not only arduous but also demand a significant amount of computational resources.