Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education?

INTRODUCTION With increasing awareness of the societal risks of algorithmic bias and encroaching automation, issues of fairness, accountability, and transparency in data-driven AI systems have received growing academic attention in multiple high-stakes contexts, including healthcare, loan-granting, and hiring (e.g., Barocas & Selbst, 2016; Holstein, Wortman Vaughan, Daumé III, Dudik, & Wallach, 2019; Veale, Van Kleek, & Binns, 2018). Given these noble intentions, why might AIEd systems have inequitable impacts? In this chapter, we ask whether AIEd systems will ultimately serve to A mplify I nequities in Ed ucation, or alternatively, whether they will help to A lleviate existing inequities. We discuss four lenses that can be used to examine how and why AIEd systems risk amplifying existing inequities: (1) factors inherent to the overall socio-technical system design; (2) the use of datasets that reflect historical inequities; (3) factors inherent to the underlying algorithms used to drive machine learning and automated decision-making, and (4) factors that emerge through a complex interplay between automated and human decision-making. Building from these lenses, we then outline possible paths towards more equitable futures for AIEd, while highlighting debates surrounding each proposal. In doing so, we hope to provoke new conversations around the design of equitable AIEd, and to push ongoing conversations in the field forward. PATHWAYS TOWARD INEQUITY IN AIED We begin by presenting four lenses to understand how AIEd systems might amplify existing inequities or even create new ones (cf. While each lens provides a different way of examining pathways towards inequity in AIEd, all are pointed at the same underlying socio-technical system. Figure 1 provides a coarse-grained overview of the broader social-technical systems in which AIEd systems are embedded, and some of the components we will refer to in the four lenses. The accumulated, collective decisions of designers, researchers, policy-makers, and other stakeholders shape these systems' designs. In addition to using or being affected by AIEd systems, on-the-ground stakeholders such as students, teachers, or school administrators may also play a role in shaping their designs; whether directly, through participatory design processes, or indirectly through the passive generation of training data while interacting with an AIEd interface. In turn, decisions regarding what data is used to shape an AIEd system's design (e.g., when used as training data for use with machine learning methods) can shape an AIEd system's algorithmic behavior (e.g., instructional policies learned from data).