Recommendation Systems with Distribution-Free Reliability Guarantees

The digitization of all manner of services has introduced recommendation systems into many aspects of our day-to-day lives. In particular, recommendation systems are now being applied to safety-critical domains such as making lifestyle recommendations to patients in healthcare [Hammer et al., 2015, Tran et al., 2021]. It is therefore increasingly important that deployed recommender systems do not output recommendations devoid of uncertainty annotations. Meaningful recommendations should come with transparent and reliable statistical assessments. To date, the majority of deployed systems have fallen far short of this desideratum [Covington et al., 2016, Liu et al., 2017, Geyik et al., 2018]. Augmenting recommendation systems with internal tracking of statistical error rates would unlock new capabilities and applications. One such capability is the ability to enforce auxiliary constraints while still guaranteeing a baseline number of high-quality items in each slate of recommendations. For example, we could diversify slates whose quality we are confident in, while leaving lower-confidence slates untouched. Furthermore, the strong guarantees provided by uncertainty quantification are a prerequisite for applying recommendation systems to safety-critical tasks such as medical diagnosis, where a misdiagnosis due to uncertain predictions can be fatal.