Goto

Collaborating Authors

 fauss


Evaluating AI-Generated Essays with GRE Analytical Writing Assessment

Zhong, Yang, Hao, Jiangang, Fauss, Michael, Li, Chen, Wang, Yuan

arXiv.org Artificial Intelligence

The recent revolutionary advance in generative AI enables the generation of realistic and coherent texts by large language models (LLMs). Despite many existing evaluation metrics on the quality of the generated texts, there is still a lack of rigorous assessment of how well LLMs perform in complex and demanding writing assessments. This study examines essays generated by ten leading LLMs for the analytical writing assessment of the Graduate Record Exam (GRE). We assessed these essays using both human raters and the e-rater automated scoring engine as used in the GRE scoring pipeline. Notably, the top-performing Gemini and GPT-4o received an average score of 4.78 and 4.67, respectively, falling between "generally thoughtful, well-developed analysis of the issue and conveys meaning clearly" and "presents a competent analysis of the issue and conveys meaning with acceptable clarity" according to the GRE scoring guideline. We also evaluated the detection accuracy of these essays, with detectors trained on essays generated by the same and different LLMs.


AI tool offers cure for scattered medical data

#artificialintelligence

A patient in the ER, ICU, and other care environments is often connected to monitoring equipment such as cardiac monitors or ventilators, which capture a range of medical data points: heart rate, respiratory rate, oxygen saturation levels, body temperature, and more. Studying these numbers over time can yield vital information about the body's physiological patterns indicating imminent deterioration such as cardiac arrests, respiratory depression, and stroke. Unfortunately, in most cases, medical professionals are not able to leverage such data because most information from medical devices is transient. Very little of the bedside device data makes its way to the EHR, and the rest is deleted once a patient is taken off of the monitor. When a patient is transferred to a different unit, there is no easy way for members of the care team to relay historical data to the new care team.