Goto

Collaborating Authors

 text readability


A study of Vietnamese readability assessing through semantic and statistical features

arXiv.org Artificial Intelligence

Determining the difficulty of a text involves assessing various textual features that may impact the reader's text comprehension, yet current research in Vietnamese has only focused on statistical features. This paper introduces a new approach that integrates statistical and semantic approaches to assessing text readability. Our research utilized three distinct datasets: the Vietnamese Text Readability Dataset (ViRead), OneStopEnglish, and RACE, with the latter two translated into Vietnamese. Advanced semantic analysis methods were employed for the semantic aspect using state-of-the-art language models such as PhoBERT, ViDeBERTa, and ViBERT. In addition, statistical methods were incorporated to extract syntactic and lexical features of the text. We conducted experiments using various machine learning models, including Support Vector Machine (SVM), Random Forest, and Extra Trees and evaluated their performance using accuracy and F1 score metrics. Our results indicate that a joint approach that combines semantic and statistical features significantly enhances the accuracy of readability classification compared to using each method in isolation. The current study emphasizes the importance of considering both statistical and semantic aspects for a more accurate assessment of text difficulty in Vietnamese. This contribution to the field provides insights into the adaptability of advanced language models in the context of Vietnamese text readability. It lays the groundwork for future research in this area.


Measuring and Modifying the Readability of English Texts with GPT-4

arXiv.org Artificial Intelligence

The success of Large Language Models (LLMs) in other domains has raised the question of whether LLMs can reliably assess and manipulate the readability of text. We approach this question empirically. First, using a published corpus of 4,724 English text excerpts, we find that readability estimates produced ``zero-shot'' from GPT-4 Turbo and GPT-4o mini exhibit relatively high correlation with human judgments (r = 0.76 and r = 0.74, respectively), out-performing estimates derived from traditional readability formulas and various psycholinguistic indices. Then, in a pre-registered human experiment (N = 59), we ask whether Turbo can reliably make text easier or harder to read. We find evidence to support this hypothesis, though considerable variance in human judgments remains unexplained. We conclude by discussing the limitations of this approach, including limited scope, as well as the validity of the ``readability'' construct and its dependence on context, audience, and goal.


Exploring Hybrid Linguistic Features for Turkish Text Readability

arXiv.org Artificial Intelligence

The integration of complex morphological, syntactic, semantic, and Automatic Readability Assessment (ARA) is an discourse features in modern ARA approaches offers important task in computational linguistics that the possibility of significantly improving the aims to automatically determine the level of difficulty current readability studies in Turkish. In this paper, of understanding a written text, which has we present the first ARA study for Turkish. Our implications for various fields, such as healthcare, study combines traditional raw text features with education, and accessibility (Vajjala, 2021). In lexical, morpho-syntactic, and syntactic information the healthcare sector, medical practitioners can use to create an advanced readability assessment ARA tools to ensure patient information and consent tool for Turkish. We demonstrate the effectiveness forms are easily understandable (Ley and Florio, of our tool on a new corpus of Turkish popular 1996). In the field of education, teachers and science magazine articles, published for different learners alike can benefit from ARA systems to age groups and educational levels. Our study adapt materials to the appropriate language proficiency aims to contribute to the development of automated level (Kintsch and Vipond, 2014).


Large Language Models and Control Mechanisms Improve Text Readability of Biomedical Abstracts

arXiv.org Artificial Intelligence

Biomedical literature often uses complex language and inaccessible professional terminologies. That is why simplification plays an important role in improving public health literacy. Applying Natural Language Processing (NLP) models to automate such tasks allows for quick and direct accessibility for lay readers. In this work, we investigate the ability of state-of-the-art large language models (LLMs) on the task of biomedical abstract simplification, using the publicly available dataset for plain language adaptation of biomedical abstracts (\textbf{PLABA}). The methods applied include domain fine-tuning and prompt-based learning (PBL) on: 1) Encoder-decoder models (T5, SciFive, and BART), 2) Decoder-only GPT models (GPT-3.5 and GPT-4) from OpenAI and BioGPT, and 3) Control-token mechanisms on BART-based models. We used a range of automatic evaluation metrics, including BLEU, ROUGE, SARI, and BERTscore, and also conducted human evaluations. BART-Large with Control Token (BART-L-w-CT) mechanisms reported the highest SARI score of 46.54 and T5-base reported the highest BERTscore 72.62. In human evaluation, BART-L-w-CTs achieved a better simplicity score over T5-Base (2.9 vs. 2.2), while T5-Base achieved a better meaning preservation score over BART-L-w-CTs (3.1 vs. 2.6). We also categorised the system outputs with examples, hoping this will shed some light for future research on this task. Our code, fine-tuned models, and data splits are available at \url{https://github.com/HECTA-UoM/PLABA-MU}


Prompt-based Learning for Text Readability Assessment

arXiv.org Artificial Intelligence

We propose the novel adaptation of a pre-trained seq2seq model for readability assessment. We prove that a seq2seq model - T5 or BART - can be adapted to discern which text is more difficult from two given texts (pairwise). As an exploratory study to prompt-learn a neural network for text readability in a text-to-text manner, we report useful tips for future work in seq2seq training and ranking-based approach to readability assessment. Specifically, we test nine input-output formats/prefixes and show that they can significantly influence the final model performance. Also, we argue that the combination of text-to-text training and pairwise ranking setup 1) enables leveraging multiple parallel text simplification data for teaching readability and 2) trains a neural model for the general concept of readability (therefore, better cross-domain generalization). At last, we report a 99.6% pairwise classification accuracy on Newsela and a 98.7% for OneStopEnglish, through a joint training approach.


Optimizing Readability Using Genetic Algorithms

arXiv.org Artificial Intelligence

It corresponds to the level of literacy that is expected from the readers in the target audience. In this way, readability is considered one of the most critical factors that facilitate the user experience when consuming information. It is crucial because it is key to establishing a trusting relationship between information producers and consumers. It must be considered that some factors, such as complexity, legibility, or typography, contribute to making a text readable. However, not all factors are quantifiable and cannot be optimized by automatic techniques. In this paper, we focus solely and exclusively on factors of a quantifiable nature, which always revolve around basic or advanced statistics associated with the text to be optimized. Therefore, text readability refers to how simple it is to read and comprehend a given text, depending on its unique characteristics. These characteristics are usually measurable through metrics like the number of syllables in a sentence.