Med-gte-hybrid: A contextual embedding transformer model for extracting actionable information from clinical texts

Besides structured data, patient care information in Electronic Health Records (EHRs) comprises data in unstructured form (i.e., clinical notes). While EHR information may overlap between structured and unstructured sections, some crucial information remains only in the unstructured sections [1-5]. Relevant information for clinical decisions can easily be overlooked when dealing with large amounts of notes. Previous investigations have already found that clinical text alone can often provide su!cient information for decisions [2, 6, 7]. However, extracting actionable information from clinical text remains di!cult due to language variability, inconsistent use of medical terminology, and lack of standardised formatting [8]. In addition, the lack of structure introduces ambiguities and inconsistencies in the text. Thus, it is still di!cult to accurately interpret and analyse clinical notes for decision support systems. As a result, the development of advanced natural language processing (NLP) models that can extract and represent information from clinical narrative has become a focal point of research in digital medicine [2, 9, 10]. Contextual embedding models have emerged as a powerful tool for transforming unstructured texts into dense vector representations to encode rich semantic information [11, 12].