Transitivity Recovering Decompositions: Interpretable and Robust Fine-Grained Relationships