Ranking systems influence decision-making in high-stakes domains like health, education, and employment, where they can have substantial economic and social impacts. This makes the integration of safety mechanisms essential. One such mechanism is $\textit{abstention}$, which enables algorithmic decision-making system to defer uncertain or low-confidence decisions to human experts. While abstention have been predominantly explored in the context of classification tasks, its application to other machine learning paradigms remains underexplored. In this paper, we introduce a novel method for abstention in pairwise learning-to-rank tasks. Our approach is based on thresholding the ranker's conditional risk: the system abstains from making a decision when the estimated risk exceeds a predefined threshold. Our contributions are threefold: a theoretical characterization of the optimal abstention strategy, a model-agnostic, plug-in algorithm for constructing abstaining ranking models, and a comprehensive empirical evaluations across multiple datasets, demonstrating the effectiveness of our approach.

Bottomline is at the forefront of digital transformation. We are a growing global market leader uniquely equipped to address the changing needs of how businesses pay and get paid. Our culture of working with and for each other enables us to delight our customers. We empower our teams to think like owners driving customer delight, helping them grow their business and win in their markets. We're looking for an experienced, ambitious and hands-on Product Management professional to define, operationalize and support product lines to commercialize our DnA Hub.

According to a McKinsey report, the global payments market is estimated to grow from $1.9 trillion in 2017 at 9% CAGR to $2.9 trillion by 2022. Portsmouth-based Bottomline Technologies (Nasdaq: EPAY) is a leading player in the B2B world of digital payments. Bottomline Technologies was set up in 1999 to provide enterprise-wide payment solutions to address the need to migrate enterprises away from pre-printed, paper checks as a payment solution. Since then, the company has come a long way. Its product suite is aimed at providing organizations with access to a single platform that can control, manage, and issue all payments across an enterprise.