science assessment
Matching Exemplar as Next Sentence Prediction (MeNSP): Zero-shot Prompt Learning for Automatic Scoring in Science Education
Wu, Xuansheng, He, Xinyu, Liu, Tianming, Liu, Ninghao, Zhai, Xiaoming
Developing models to automatically score students' written responses to science problems is critical for science education. However, collecting and labeling sufficient student responses for training models is time and cost-consuming. Recent studies suggest that pre-trained language models can be adapted to downstream tasks without fine-tuning with prompts. However, no research has employed such a prompt approach in science education. As student responses are presented with natural language, aligning the scoring procedure as the next sentence prediction task using prompts can skip the costly fine-tuning stage. In this study, we developed a zero-shot approach to automatically score student responses via Matching Exemplars as Next Sentence Prediction (MeNSP). This approach employs no training samples. We first apply MeNSP in scoring three assessment tasks of scientific argumentation and found machine-human scoring agreements, Cohen's Kappa ranges from 0.30 to 0.57, and F1 score ranges from 0.54 to 0.81. To improve the performance, we extend our research to the few-shots setting, either randomly selecting labeled student responses or manually constructing responses to fine-tune the models. We find that one task's performance is improved with more samples, Cohen's Kappa from 0.30 to 0.38, and F1 score from 0.54 to 0.59; for the two others, scoring performance is not improved. We also find that randomly selected few-shots perform better than the human expert-crafted approach. This study suggests that MeNSP can yield referable automatic scoring for student responses while significantly reducing the cost of model training. This method can benefit low-stakes classroom assessment practices in science education. Future research should further explore the applicability of the MeNSP in different types of assessment tasks in science education and improve the model performance.
Applying artificial intelligence to science education
A new review published in the Journal of Research in Science Teaching highlights the potential of machine learning--a subset of artificial intelligence--in science education. Although the authors initiated their review before the COVID-19 outbreak, the pandemic highlights the need to examine cutting-edge digital technologies as we re-think the future of teaching and learning. Based on a review of 47 studies, investigators developed a framework to conceptualize machine learning applications in science assessment. The article aims to examine how machine learning has revolutionized the capacity of science assessment in terms of tapping into complex constructs, improving assessment functionality, and facilitating scoring automaticity. Based on their investigation, the researchers identified various ways in which machine learning has transformed traditional science assessment, as well as anticipated impacts that it will likely have in the future (such as providing personalized science learning and changing the process of educational decision-making).
Applying artificial intelligence to science education
A new review published in the Journal of Research in Science Teaching highlights the potential of machine learning--a subset of artificial intelligence--in science education. Although the authors initiated their review before the COVID-19 outbreak, the pandemic highlights the need to examine cutting-edge digital technologies as we re-think the future of teaching and learning. Based on a review of 47 studies, investigators developed a framework to conceptualize machine learning applications in science assessment. The article aims to examine how machine learning has revolutionized the capacity of science assessment in terms of tapping into complex constructs, improving assessment functionality, and facilitating scoring automaticity. Based on their investigation, the researchers identified various ways in which machine learning has transformed traditional science assessment, as well as anticipated impacts that it will likely have in the future (such as providing personalized science learning and changing the process of educational decision-making).
Don't abandon evidence and process on air pollution policy
Air pollution kills--scientists have known this for many years. But how do they know? The global scientific community has developed and agreed upon a framework that draws on multiple lines of evidence across different scientific disciplines to assess the existence and strength of links between air pollution and health. In the United States, federal policies require use of this science-based framework to ensure that air pollution standards protect the public's health. But now this science-based policy process--and public health--are at risk.