Unfair Utilities and First Steps Towards Improving Them

A challenge in algorithmic fairness is to formalize the notion of fairness. Often, one attribute S is considered protected (also called sensitive) and a quantity Y is to be predicted as Ŷ from some covariates X. Many criteria for fairness correspond to constraints on the joint distribution of (S,X,Y,Ŷ) that can often be phrased as (conditional) independence statements or take the causal structure of the problem into account [see, for example, Barocas et al., 2023, Verma and Rubin, 2018, Nilforoshan et al., 2022, for an overview]. In this work, we propose an alternative point of view that considers situations where an agent aims to optimize a policy as to maximize a known utility. In such scenarios, unwanted discrimination may occur if the utility itself is unfair.