Context is Enough: Empirical Validation of $\textit{Sequentiality}$ on Essays
Sunny, Amal, Gupta, Advay, Sreekumar, Vishnu
–arXiv.org Artificial Intelligence
Recent work has proposed using Large Language Models (LLMs) to quantify narrative flow through a measure called sequentiality, which combines topic and contextual terms. A recent critique argued that the original results were confounded by how topics were selected for the topic-based component, and noted that the metric had not been validated against ground-truth measures of flow. That work proposed using only the contextual term as a more conceptually valid and interpretable alternative. In this paper, we empirically validate that proposal. Using two essay datasets with human-annotated trait scores, ASAP++ and ELLIPSE, we show that the contextual version of sequentiality aligns more closely with human assessments of discourse-level traits such as Organization and Cohesion. While zero-shot prompted LLMs predict trait scores more accurately than the contextual measure alone, the contextual measure adds more predictive value than both the topic-only and original sequentiality formulations when combined with standard linguistic features. Notably, this combination also outperforms the zero-shot LLM predictions, highlighting the value of explicitly modeling sentence-to-sentence flow. Our findings support the use of context-based sequentiality as a validated, interpretable, and complementary feature for automated essay scoring and related NLP tasks.
arXiv.org Artificial Intelligence
Nov-13-2025
- Country:
- Asia
- Japan > Kyūshū & Okinawa
- Kyūshū > Miyazaki Prefecture > Miyazaki (0.04)
- Thailand > Bangkok
- Bangkok (0.04)
- Japan > Kyūshū & Okinawa
- Europe > Spain
- Catalonia > Barcelona Province > Barcelona (0.04)
- North America
- Mexico > Mexico City
- Mexico City (0.04)
- United States > Texas
- Travis County > Austin (0.04)
- Mexico > Mexico City
- Asia
- Genre:
- Research Report > New Finding (1.00)
- Technology: