A Capabilities Approach to Studying Bias and Harm in Language Technologies

In moving from excluding the majority of the world's languages to blindly adopting what we make for English, we first risk importing the same harms we have at best mitigated and at least measured for English. For instance, Yong et al. [15] showed how prompting GPT-4 in low-resource languages circumvents guardrails that are effective in English. However, in evaluating and mitigating harms arising from adopting new technologies into such contexts, we often disregard (1) the actual community needs of Language Technologies, and (2) biases and fairness issues within the context of the communities. Here, we consider fairness, bias, and inclusion in Language Technologies through the lens of the Capabilities Approach [12]. The Capabilities Approach centers what people are capable of achieving, given their intersectional social, political, and economic contexts instead of what resources are (theoretically) available to them. In the following sections, we detail the Capabilities Approach, its relationship to multilingual and multicultural evaluation, and how the framework affords meaningful collaboration with community members in defining and measuring harms of Language Technologies. 2 THE CAPABILITIES APPROACH The Capabilities Approach is a framework in developmental economic studies proposed by Amartya Sen in a series of articles published as far back as 1974 [1]. It has been applied to varied fields including environmental justice [e.g.