Algorithmic Detection of Rank Reversals, Transitivity Violations, and Decomposition Inconsistencies in Multi-Criteria Decision Analysis

Our work focuses on providing a mechanism capable of measuring the performance of a MCDM on a given set of alternatives, with the collateral goal of building a global ranking of the e ffectiveness of di fferent MCDMs. We have implemented these tests within the open-source Scikit-Criteria library, leveraging its RankResult and RanksComparator data structures as fundamental building blocks for comparative ranking analysis. RRT1 systematically evaluates the stability of the optimal alternative when suboptimal alternatives are degraded, employing a controlled mutation strategy and providing comprehensive documentation of the experimental context. This approach provides decision analysts with the following: 1. Quantitative stability assessment: Precise measures of how often methods exhibit rank reversal 2. Sensitivity mapping: Identification of which alternatives and criteria are most prone to instability 3. Method comparison: Objective basis for comparing the robustness of di fferent MCDA approaches 4. Confidence intervals: Statistical bounds on decision reliability through repeated experimentation The algorithm addresses the complications that arise from preprocessing pipelines that can eliminate alternatives, ensuring "graceful degradation" by assigning appropriate worst ranks to maintain completeness.