Enhancing Clinical Text Classification via Fine-Tuned DRAGON Longformer Models

This study explores the optimization of the DRAGON Longformer base model for clinical text classification, specifically targeting the binary classification of medical case descriptions. A dataset of 500 clinical cases containing structured medical observations was used, with 400 cases for training and 100 for validation. Enhancements to the pre - trained joeranbosma/dragon - longformer - base - mixed - domain model included hyperparameter tuning, domain - specific preprocessing, and architectural adjustments. Key modifications involved increasing sequence length from 512 to 1024 tokens, adjusting learning rates from 1e - 05 to 5e - 06, extending training epochs from 5 to 8, and incorporating specialized medical terminology. The optimized model achieved notable performance gains: accuracy improved from 72.0% to 85.2%, precision from 68.0% to 84.1%, recall from 75.0% to 86.3%, and F1 - score from 71.0% to 85.2%. Statistical analysis confirmed the significance of these improvements (p < .001). The model demonstrated enhanced capability in interpreting medical terminology, anatomical measurements, and clinical observations. These findings contribute to domain - specific language model research and offer practical implications for clinical natural language processing applications. The optimized model ' s strong performance across diverse medical conditions underscores its potential for broad use in healthcare settings. Enhancing Clinical Text Classification via Fine - Tuned DRAGON Longformer Models Introduction Natural language processing (NLP) in healthcare has continued to advance rapidly, revolutionizing the ability to analyze clinical texts and automate the extraction of valuable insights from massive amounts of medical documentation (Khurana, Koli, Khatter, & Singh, 2023). Over the past few years, large language models (LLMs) have emerged as powerful tools for gaining insight from and processing clinical narratives, creating capabilities that have never been seen before in medical text classification, entity recognition, and clinical decision support (Wang et al., 2018). The DRAGON (Deep Representation Analysis for General - domain Ontology Networks) framework was a specialized version of medical text processing out of all these models (Bosma et al., 2025). Beltagy, Peters, and Cohan (2020) state that the DRAGON longformer model, built on top of the Longformer architecture, addresses the quadratic computational complexity issue of traditional transformer models by processing long sequences.