Large-scale randomized experiment reveals machine learning helps people learn and remember more effectively

The greater degree of control and personalization offered by learning apps and online platforms promise to facilitate the design and implementation of automated, data-driven teaching policies that adapt to each learner's knowledge over time, improving upon the traditional one-size-fits-all human instruction. However, to fulfill this promise, it is necessary to develop adaptive data-driven models of the learners, which accurately quantify their knowledge, and efficient methods to find teaching policies that are provably optimal under the learners' models [1, 2]. In this context, research in the (theoretical) computer science literature has been typically focused on finding teaching policies that enjoy optimality guarantees under simplified mathematical models of the learner's knowledge [3-7]. In contrast, research in cognitive sciences has focused on measuring the effectivity of a variety of heuristic teaching policies informed by psychologically valid models of the learner's knowledge using (small) randomized control trials [8-11]. Only very recently, Tabibian et al. [12] has introduced a machine learning modeling framework that bridges the gap between both lines of research--their framework can be used to determine the optimal rate of study a learner should follow under a model of the learner's memory state that is informed by real human memory data. However, in the evaluation of their framework, the authors resort to a natural experiment using data from a popular language-learning online platform rather than a randomized control trial, the gold standard in the cognitive sciences literature. As a result, it has been argued that, in an interventional setting, an actual learner following the rate of study may fail to achieve optimal performance [1]. In this paper, we build upon the modeling framework of Tabibian et al. [12] and design Select, a simple, efficient and adaptive machine learning algorithm with theoretical guarantees to determine which questions to include in a learner's sessions of study over time, rather than optimizing the rate of study as in Tabibian et al.,