Inference for an Algorithmic Fairness-Accuracy Frontier

Algorithms are increasingly used in many aspects of life, often to guide or support high stake decisions. For example, algorithms are used to predict criminal re-offense risk, and this prediction feeds into the determination of which defendants should receive bail; to predict a job market candidate's productivity, and this prediction feeds into hiring decisions; to predict an applicant's likelihood of default on a loan, and this prediction feeds into the decision of who should receive the loan; to predict a student's performance in college, and this prediction feeds into the decision of which students should be admitted to college; and to assign a health risk score to a patient, and this score feeds into the decision of which patients to treat. Yet, a growing body of literature documents that algorithms may exhibit bias against legally protected subgroups of the population, both in terms of their ability to make correct predictions, and in the type of decisions that they lead to (see, e.g., Angwin et al., 2016, Arnold et al., 2021, Obermeyer et al., 2019, Berk et al., 2021). The bias may arise, for example, because of the choice of labels in the data that the algorithm is trained on, the objective function that the algorithm optimizes, the training procedure, and various other factors involved in the construction of the algorithm. To understand what drives algorithmic bias, several models have been put forth that decompose the source of disparity (e.g., Rambachan et al., 2020a) or account for taste-based discrimination and unobservables in the generation of training labels (e.g., Rambachan and Roth, 2020). Regardless of whether the screening decision is based on a prediction made by a human or by an algorithm, the law recognizes two main categories of discrimination: disparate treatment, which amounts to deliberately treating an individual differently based on their membership to a protected class; and disparate impact, which amounts to adversely affecting a protected class disproportionately, no matter the intent (see, e.g., Kleinberg et al., 2018b, Blattner and Spiess, 2022, for a review of the discrimination law in the U.S). Often, as part of an effort to avoid disparate treatment, algorithms are designed so that they do not take race, gender, or other sensitive attributes as an input. Even class-blind algorithms, however, may yield disparate outcome. Crucially, there are trade-offs in the design of an algorithm between making it more fair, in the sense that it has lower disparate impact, and making it more accurate, in the sense 3 that, e.g., it has a higher probability to assign treatment to the individuals that benefit from it and to not assign it to the other individuals. Indeed, improving fairness often comes at the cost of accuracy.