The Fairness-Accuracy Pareto Front

Ethical concerns regarding artificial intelligence has led to increased self-scrutiny from the machine learning community. Algorithmic fairness has proved to be a challenging research area. For one, a broadly appealing definition of fairness has long eluded philosophers, social scientists, and, more recently, the machine learning community. Currently, finding mathematical formulations of fairness is an active area of research in algorithmic fairness [Dwork et al., 2012, Chouldechova, 2016, Joseph et al., 2016]. While it is easy to agree on what is unfair, it is much harder to agree on what, exactly, is fair. For instance, ProPublica's eponymous article on machine bias [Angwin et al., 2016] uncovered prejudice against African-Americans in COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), a recidivism prediction tool developed by Northpointe. But it turns out if different fairness criteria are used than those used in the ProPublica investigation, COMPAS can be more favourably viewed [Dieterich et al., 2016, Corbett-Davies et al., 2017]. This type of dissent is unavoidable as several works have shown that certain fairness criteria cannot be simultaneously satisfied [Hardt et al., 2016, Kleinberg, 2018]. Leaving aside for now the debate over the correct definition of fairness, most works in algorithmic fairness are quite straightforward once a fairness measure is settled on.