Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA
Karim, A H M Rezaul, Uzuner, Ozlem
–arXiv.org Artificial Intelligence
Medical Visual Question Answering (MedVQA) enables natural language queries over medical images to support clinical decision-making and patient care. The MEDIQA-WV 2025 shared task addressed wound-care VQA, requiring systems to generate free-text responses and structured wound attributes from images and patient queries. We present the MasonNLP system, which employs a general-domain, instruction-tuned large language model with a retrieval-augmented generation (RAG) framework that incorporates textual and visual examples from in-domain data. This approach grounds outputs in clinically relevant exemplars, improving reasoning, schema adherence, and response quality across dBLEU, ROUGE, BERTScore, and LLM-based metrics. Our best-performing system ranked 3rd among 19 teams and 51 submissions with an average score of 41.37%, demonstrating that lightweight RAG with general-purpose LLMs -- a minimal inference-time layer that adds a few relevant exemplars via simple indexing and fusion, with no extra training or complex re-ranking -- provides a simple and effective baseline for multimodal clinical NLP tasks.
arXiv.org Artificial Intelligence
Oct-17-2025
- Country:
- Asia (0.28)
- Genre:
- Research Report > New Finding (0.46)
- Industry:
- Health & Medicine > Diagnostic Medicine > Imaging (1.00)
- Technology: