Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners

Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners Michael Vaccaro Jr. Abstract Large language models (LLMs), including OpenAI's GPT-series, have made significant advancements in recent years. Known for their expertise across diverse subject areas and quick adaptability to user-provided prompts, LLMs hold unique potential as Personalized Learning (PL) tools. Despite this potential, their application in K-12 education remains largely unexplored. This paper presents one of the first randomized controlled trials (n = 23) to evaluate the effectiveness of GPT-4 in personalizing educational science texts for middle school students. In this study, GPT-4 was used to profile student learning preferences based on choices made during a training session. For the experimental group, GPT-4 was used to rewrite science texts to align with the student's predicted profile while, for students in the control group, texts were rewritten to contradict their learning preferences. The results of a Mann-Whitney U test showed that students significantly preferred (at the.10 level) the rewritten texts when they were aligned with their profile (p =.059). These findings suggest that GPT-4 can effectively interpret and tailor educational content to diverse learner preferences, marking a significant advancement in PL technology. The limitations of this study and ethical considerations for using artificial intelligence in education are also discussed. Keywords: Large Language Models (LLMs), GPT-4, Personalized Learning, AI Generated Content (AIGC), Randomized Controlled Trial (RCT), K-12 Education 1 Introduction In 2008, the National Academy of Engineering named advancements in Personalized Learning (PL) one of the fourteen grand challenges for the twenty-first century (National Academy of Engineering, 2008). Since this time, PL has emerged as a prominent area of education research. Through this work, PL has evolved into a broad term which now encompasses a vast number of interventions and programs (Shemshack and Spector, 2020; Walkington and Bernacki, 2020). The work presented in this paper aims to build on this existing research by investigating the potential of novel Large Language Models (LLMs) to foster highly adaptive PL environments.