pharmacist
Human-Level and Beyond: Benchmarking Large Language Models Against Clinical Pharmacists in Prescription Review
Yang, Yan, Bian, Mouxiao, Li, Peiling, Wen, Bingjian, Chen, Ruiyao, Mao, Kangkun, Ye, Xiaojun, Li, Tianbin, Chen, Pengcheng, Han, Bing, Xu, Jie, Qiu, Kaifeng, Wu, Junyan
The rapid advancement of large language models (LLMs) has accelerated their integration into clinical decision support, particularly in prescription review. To enable systematic and fine-grained evaluation, we developed RxBench, a comprehensive benchmark that covers common prescription review categories and consolidates 14 frequent types of prescription errors drawn from authoritative pharmacy references. RxBench consists of 1,150 single-choice, 230 multiple-choice, and 879 short-answer items, all reviewed by experienced clinical pharmacists. We benchmarked 18 state-of-the-art LLMs and identified clear stratification of performance across tasks. Notably, Gemini-2.5-pro-preview-05-06, Grok-4-0709, and DeepSeek-R1-0528 consistently formed the first tier, outperforming other models in both accuracy and robustness. Comparisons with licensed pharmacists indicated that leading LLMs can match or exceed human performance in certain tasks. Furthermore, building on insights from our benchmark evaluation, we performed targeted fine-tuning on a mid-tier model, resulting in a specialized model that rivals leading general-purpose LLMs in performance on short-answer question tasks. The main contribution of RxBench lies in establishing a standardized, error-type-oriented framework that not only reveals the capabilities and limitations of frontier LLMs in prescription review but also provides a foundational resource for building more reliable and specialized clinical tools.
- North America > United States (0.14)
- Asia > China > Shanghai > Shanghai (0.04)
- Asia > China > Guangdong Province (0.04)
- Research Report > Experimental Study (1.00)
- Research Report > New Finding (0.69)
Pharmacist: Safety Alignment Data Curation for Large Language Models against Harmful Fine-tuning
Liu, Guozhi, Mu, Qi, Huang, Tiansheng, Wang, Xinhua, Shen, Li, Lin, Weiwei, Li, Zhang
Harmful fine-tuning issues present significant safety challenges for fine-tuning-as-a-service in large language models. Existing alignment-stage defenses, e.g., Vaccine, Repnoise, Booster, and T-Vaccine, mitigate harmful fine-tuning issues by enhancing the model's robustness during the alignment phase. While these methods have been proposed to mitigate the issue, they often overlook a critical upstream factor: the role of the original safety-alignment data. We observe that their defense performance and computational efficiency remain constrained by the quality and composition of the alignment dataset. To address this limitation, we propose Pharmacist, a safety alignment data curation solution that enhances defense against harmful fine-tuning by selecting a high-quality and safety-critical core subset from the original alignment data. The core idea of Pharmacist is to train an alignment data selector to rank alignment data. Specifically, up-ranking high-quality and safety-critical alignment data, down-ranking low-quality and non-safety-critical data. Empirical results indicate that models trained on datasets selected by Pharmacist outperform those trained on datasets selected by existing selection methods in both defense and inference performance. In addition, Pharmacist can be effectively integrated with mainstream alignment-stage defense methods. For example, when applied to RepNoise and T-Vaccine, using the dataset selected by Pharmacist instead of the full dataset leads to improvements in defense performance by 2.60\% and 3.30\%, respectively, and enhances inference performance by 3.50\% and 1.10\%. Notably, it reduces training time by 56.83\% and 57.63\%, respectively. Our code is available at https://github.com/Lslland/Pharmacist.
- Asia > China > Guangdong Province > Guangzhou (0.04)
- Asia > China > Guangdong Province > Shenzhen (0.04)
- North America > United States (0.04)
- Asia > China > Shandong Province (0.04)
- Health & Medicine > Therapeutic Area > Vaccines (0.74)
- Health & Medicine > Therapeutic Area > Immunology (0.64)
Using role-play and Hierarchical Task Analysis for designing human-robot interaction
Wingren, Mattias, Andersson, Sören, Rosenberg, Sara, Andtfolk, Malin, Hägglund, Susanne, Arachchige, Prashani Jayasingha, Nyholm, Linda
We present the use of two methods we believe warrant more use than they currently have in the field of human-robot interaction: role-play and Hierarchical Task Analysis. Some of its potential is showcased through our use of them in an ongoing research project which entails developing a robot application meant to assist at a community pharmacy. The two methods have provided us with several advantages. The role-playing provided a controlled and adjustable environment for understanding the customers' needs where pharmacists could act as models for the robot's behavior; and the Hierarchical Task Analysis ensured the behavior displayed was modelled correctly and aided development through facilitating co-design. Future research could focus on developing task analysis methods especially suited for social robot interaction.
- North America > United States > New York > New York County > New York City (0.05)
- North America > United States > Florida > Palm Beach County > Boca Raton (0.04)
- Europe > Norway > Western Norway > Rogaland > Stavanger (0.04)
- (10 more...)
medicX-KG: A Knowledge Graph for Pharmacists' Drug Information Needs
Farrugia, Lizzy, Azzopardi, Lilian M., Debattista, Jeremy, Abela, Charlie
The role of pharmacists is evolving from medicine dispensing to delivering comprehensive pharmaceutical services within multidisciplinary healthcare teams. Central to this shift is access to accurate, up-to-date medicinal product information supported by robust data integration. Leveraging artificial intelligence and semantic technologies, Knowledge Graphs (KGs) uncover hidden relationships and enable data-driven decision-making. This paper presents medicX-KG, a pharmacist-oriented knowledge graph supporting clinical and regulatory decisions. It forms the semantic layer of the broader medicX platform, powering predictive and explainable pharmacy services. medicX-KG integrates data from three sources, including, the British National Formulary (BNF), DrugBank, and the Malta Medicines Authority (MMA) that addresses Malta's regulatory landscape and combines European Medicines Agency alignment with partial UK supply dependence. The KG tackles the absence of a unified national drug repository, reducing pharmacists' reliance on fragmented sources. Its design was informed by interviews with practicing pharmacists to ensure real-world applicability. We detail the KG's construction, including data extraction, ontology design, and semantic mapping. Evaluation demonstrates that medicX-KG effectively supports queries about drug availability, interactions, adverse reactions, and therapeutic classes. Limitations, including missing detailed dosage encoding and real-time updates, are discussed alongside directions for future enhancements.
- Europe > Middle East > Malta > Eastern Region > Northern Harbour District > Msida (0.04)
- Europe > Middle East > Cyprus (0.04)
- Europe > Ireland (0.04)
- (2 more...)
- Research Report > Experimental Study (1.00)
- Research Report > New Finding (0.93)
- Questionnaire & Opinion Survey (0.92)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Ontologies (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Text Processing (1.00)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Semantic Networks (0.94)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.46)
Rx Strategist: Prescription Verification using LLM Agents System
Van, Phuc Phan, Minh, Dat Nguyen, Ngoc, An Dinh, Thanh, Huy Phan
To protect patient safety, modern pharmaceutical complexity demands strict prescription verification. We offer a new approach - Rx Strategist - that makes use of knowledge graphs and different search strategies to enhance the power of Large Language Models (LLMs) inside an agentic framework. This multifaceted technique allows for a multi-stage LLM pipeline and reliable information retrieval from a custom-built active ingredient database. Different facets of prescription verification, such as indication, dose, and possible drug interactions, are covered in each stage of the pipeline. We alleviate the drawbacks of monolithic LLM techniques by spreading reasoning over these stages, improving correctness and reliability while reducing memory demands. Our findings demonstrate that Rx Strategist surpasses many current LLMs, achieving performance comparable to that of a highly experienced clinical pharmacist. In the complicated world of modern medications, this combination of LLMs with organized knowledge and sophisticated search methods presents a viable avenue for reducing prescription errors and enhancing patient outcomes.
- Health & Medicine > Therapeutic Area > Nephrology (1.00)
- Health & Medicine > Therapeutic Area > Cardiology/Vascular Diseases (1.00)
- Health & Medicine > Pharmaceuticals & Biotechnology (1.00)
- Health & Medicine > Therapeutic Area > Endocrinology > Diabetes (0.47)
Pharmacist GOP Rep Carter urges Biden to take cognitive test in letter to White House: 'Fragile mental state'
Dr. Ronny Jackson joins'Sunday Morning Futures' to discuss his five letters asking President Biden to submit to a cognitive test and his next letter addressed to Biden's physician and the Cabinet. EXCLUSIVE: Rep. Earl "Buddy" Carter, R-Ga., wrote a letter to the White House on Monday calling on President Biden to take a cognitive assessment over concerns about his "fragile mental state" and ability to uphold his duties. In a letter obtained exclusively by Fox News Digital, Carter, who is also a pharmacist, wrote to White House Chief of Staff Jeff Zients expressing "serious concern" with Biden's cognitive state and "ability to execute the duties of the Presidency." "After numerous examples of the President's declining mental acuity, it is imperative that the White House remains transparent about the President of the United States' honest ability to uphold the duties of the office to which he swore an oath," Carter wrote. This comes after a recent report from The Wall Street Journal stating that the 81-year-old president was showing signs of poor cognitive performance in private meetings with congressional lawmakers, including by closing his eyes for extended periods, speaking so softly at times that people struggled to hear him and forgetting details about his own energy policy.
- Europe > Germany (0.15)
- North America > United States > Texas (0.05)
- Europe > Ukraine (0.05)
KorMedMCQA: Multi-Choice Question Answering Benchmark for Korean Healthcare Professional Licensing Examinations
Kweon, Sunjun, Choi, Byungjin, Kim, Minkyu, Park, Rae Woong, Choi, Edward
We introduce KorMedMCQA, the first Korean multiple-choice question answering (MCQA) benchmark derived from Korean healthcare professional licensing examinations, covering from the year 2012 to year 2023. This dataset consists of a selection of questions from the license examinations for doctors, nurses, and pharmacists, featuring a diverse array of subjects. We conduct baseline experiments on various large language models, including proprietary/open-source, multilingual/Korean-additional pretrained, and clinical context pretrained models, highlighting the potential for further enhancements. We make our data publicly available on HuggingFace (https://huggingface.co/datasets/sean0042/KorMedMCQA) and provide a evaluation script via LM-Harness, inviting further exploration and advancement in Korean healthcare environments.
Development and Testing of a Novel Large Language Model-Based Clinical Decision Support Systems for Medication Safety in 12 Clinical Specialties
Ong, Jasmine Chiat Ling, Jin, Liyuan, Elangovan, Kabilan, Lim, Gilbert Yong San, Lim, Daniel Yan Zheng, Sng, Gerald Gui Ren, Ke, Yuhe, Tung, Joshua Yi Min, Zhong, Ryan Jian, Koh, Christopher Ming Yao, Lee, Keane Zhi Hao, Chen, Xiang, Chng, Jack Kian, Than, Aung, Goh, Ken Junyang, Ting, Daniel Shu Wei
Importance: We introduce a novel Retrieval Augmented Generation (RAG)-Large Language Model (LLM) as a Clinical Decision Support System (CDSS) for safe medication prescription. This model addresses the limitations of traditional rule-based CDSS by providing relevant prescribing error alerts tailored to patient context and institutional guidelines. Objective: The study evaluates the efficacy of an LLM-based CDSS in identifying medication errors across various medical and surgical case vignettes, compared to a human expert panel. It also examines clinician preferences among different CDSS integration modalities: junior pharmacist, LLM-based CDSS alone, and a combination of both. Design, Setting, and Participants: Utilizing a RAG model with GPT-4.0, the study involved 61 prescribing error scenarios within 23 clinical vignettes across 12 specialties. An expert panel assessed these cases using the PCNE classification and NCC MERP index. Three junior pharmacists independently reviewed each vignette under simulated conditions. Main Outcomes and Measures: The study assesses the LLM-based CDSS's accuracy, precision, recall, and F1 scores in identifying Drug-Related Problems (DRPs), compared to junior pharmacists alone or in an assistive mode with the CDSS. Results: The co-pilot mode of RAG-LLM significantly improved DRP identification accuracy by 22% over solo pharmacists. It showed higher recall and F1 scores, indicating better detection of severe DRPs, despite a slight decrease in precision. Accuracy varied across categories when pharmacists had access to RAG-LLM responses. Conclusions: The RAG-LLM based CDSS enhances medication error identification accuracy when used with junior pharmacists, especially in detecting severe DRPs.
- North America > United States (0.28)
- Asia > Singapore > Central Region > Singapore (0.04)
- Europe > United Kingdom > England (0.04)
- (5 more...)
- Research Report > Experimental Study (1.00)
- Research Report > Strength High (0.93)
- Research Report > New Finding (0.87)
- Health & Medicine > Therapeutic Area > Urology (1.00)
- Health & Medicine > Therapeutic Area > Oncology (1.00)
- Health & Medicine > Therapeutic Area > Nephrology (1.00)
- (14 more...)
Human-machine cooperation: optimization of drug retrieval sequencing in automated drug dispensing systems
Yuan, Mengge, Wu, Kan, Zhao, Ning
Automated drug dispensing systems (ADDSs) are increasingly in demand in today's pharmacies, primarily driven by the growing ageing population. Recognizing the practical challenges faced by pharmacies implementing ADDSs, this study aims to optimize the layout design and sequencing issues within a human-machine cooperation environment to enhance the system throughput of ADDSs. Specifically, we develop models for drug retrieval sequencing under different system layout designs, taking into account the stochastic sorting time of pharmacists. The prescription order arrival pattern follows a successive arrival mode. To assess the efficiency of ADDSs with one input/output point and two input/output points, we propose dual command retrieval sequencing models that optimize the retrieval sequence of drugs in adjacent prescription orders. Notably, our models incorporate the stochastic sorting time of pharmacists to analyze its impact on ADDS performance. Through experimental comparisons of average picking times for prescription orders under various operational conditions, we demonstrate that a system layout design incorporating two input/output points significantly enhances the efficiency of prescription order fulfilment within a human-machine cooperation environment. Furthermore, our proposed retrieval sequencing method outperforms dynamic programming, greedy, and random strategies in terms of improving prescription order-picking efficiency. By addressing the layout design and sequencing challenges, our research contributes to the field of intelligent warehousing, particularly in smart pharmacies. The findings provide valuable insights for healthcare facilities and organizations seeking to optimize ADDS performance and enhance drug dispensing efficiency.
ABiMed: An intelligent and visual clinical decision support system for medication reviews and polypharmacy management
Mouazer, Abdelmalek, Léguillon, Romain, Boudegzdame, Nada, Levrard, Thibaud, Bars, Yoann Le, Simon, Christian, Séroussi, Brigitte, Grosjean, Julien, Lelong, Romain, Letord, Catherine, Darmoni, Stéfan, Schuers, Matthieu, Sedki, Karima, Dubois, Sophie, Falcoff, Hector, Tsopra, Rosy, Lamy, Jean-Baptiste
Background: Polypharmacy, i.e. taking five drugs or more, is both a public health and an economic issue. Medication reviews are structured interviews of the patient by the community pharmacist, aiming at optimizing the drug treatment and deprescribing useless, redundant or dangerous drugs. However, they remain difficult to perform and time-consuming. Several clinical decision support systems were developed for helping clinicians to manage polypharmacy. However, most were limited to the implementation of clinical practice guidelines. In this work, our objective is to design an innovative clinical decision support system for medication reviews and polypharmacy management, named ABiMed. Methods: ABiMed associates several approaches: guidelines implementation, but the automatic extraction of patient data from the GP's electronic health record and its transfer to the pharmacist, and the visual presentation of contextualized drug knowledge using visual analytics. We performed an ergonomic assessment and qualitative evaluations involving pharmacists and GPs during focus groups and workshops. Results: We describe the proposed architecture, which allows a collaborative multi-user usage. We present the various screens of ABiMed for entering or verifying patient data, for accessing drug knowledge (posology, adverse effects, interactions), for viewing STOPP/START rules and for suggesting modification to the treatment. Qualitative evaluations showed that health professionals were highly interested by our approach, associating the automatic guidelines execution with the visual presentation of drug knowledge. Conclusions: The association of guidelines implementation with visual presentation of knowledge is a promising approach for managing polypharmacy. Future works will focus on the improvement and the evaluation of ABiMed.
- Europe > France > Normandy > Seine-Maritime > Rouen (0.05)
- Europe > France > Île-de-France > Paris > Paris (0.04)
- North America > Canada (0.04)
- (2 more...)