Groza, Tudor
An evaluation of GPT models for phenotype concept recognition
Groza, Tudor, Caufield, Harry, Gration, Dylan, Baynam, Gareth, Haendel, Melissa A, Robinson, Peter N, Mungall, Christopher J, Reese, Justin T
Objective: Clinical deep phenotyping and phenotype annotation play a critical role in both the diagnosis of patients with rare disorders as well as in building computationally-tractable knowledge in the rare disorders field. These processes rely on using ontology concepts, often from the Human Phenotype Ontology, in conjunction with a phenotype concept recognition task (supported usually by machine learning methods) to curate patient profiles or existing scientific literature. With the significant shift in the use of large language models (LLMs) for most NLP tasks, we examine the performance of the latest Generative Pre-trained Transformer (GPT) models underpinning ChatGPT as a foundation for the tasks of clinical phenotyping and phenotype annotation. Materials and Methods: The experimental setup of the study included seven prompts of various levels of specificity, two GPT models (gpt-3.5-turbo and gpt-4.0) and two established gold standard corpora for phenotype recognition, one consisting of publication abstracts and the other clinical observations. Results: Our results show that, with an appropriate setup, these models can achieve state of the art performance. The best run, using few-shot learning, achieved 0.58 macro F1 score on publication abstracts and 0.75 macro F1 score on clinical observations, the former being comparable with the state of the art, while the latter surpassing the current best in class tool. Conclusion: While the results are promising, the non-deterministic nature of the outcomes, the high cost and the lack of concordance between different runs using the same prompt and input make the use of these LLMs challenging for this particular task.
From Personal Notes to Linked Social Media
Dragan, Laura (National University of Ireland, Galway) | Passant, Alexandre (National University of Ireland, Galway) | Groza, Tudor (National University of Ireland, Galway) | Handschuh, Siegfried (National University of Ireland, Galway)
Semantic technologies are available, and gain popularity on the Web as well as on the desktop, but both (desktop and Web) act as large data silos, making personal and online data difficult to interlink. We propose a system that enables easy publishing of personal notes as linked social media content, while at the same time semantically enriching the desktop resources with information retrieved from the Linked Data cloud. The transformation, publication and linking process is integrated with the familiar desktop applications and online blogging platforms, providing a better usability experience.