Scalable and Interpretable Representation Alignment with Ordinal Similarity

Evaluating representation similarity is fundamental to representation learning. However, existing metrics suffer from significant limitations: they lack interpretability due to shifting baselines, lack robustness to outliers, and are computationally intractable for large datasets, forcing reliance on heuristic approximations. To address this, we develop an ordinal-similarity framework, instan-Figure 1. TSI and QSI measure alignment between two representiated by the Triplet (TSI) and Quadruplet (QSI) tation spaces (e.g., Visual and Textual) by quantifying the conSimilarity Indices, which measure alignment bysistency of ordinal relationships. TSI checks if relative similarity quantifying the consistency of ordinal relation-from an anchor is preserved (e.g., 'Is Acloser to B than to C?'). QSI compares relative similarity between distinct pairs (e.g., 'Is A ships. We theoretically demonstrate this formu-closer to B than C is to D?') lation is inherently interpretable, robust to outliers, and computationally efficient. Finally, wemodel design and behavioral analysis, the reliability of these establish a formal equivalence between TSI andmetrics is paramount for the interpretability of increasingly local neighborhood alignment, measured by Mu-ubiquitous AI systems.