Resource-Efficient Fine-Tuning of LLaMA-3.2-3B for Medical Chain-of-Thought Reasoning

Abstract--Large Language Models (LLMs) are increasingly being applied in healthcare, but challenges in reasoning transparency, factual consistency, and domain-specific adaptability limit their safe deployment in clinical settings. This paper presents a proof-of-concept study on the fine-tuning of Meta's LLaMA-3.2 (3B Instruct) for the medical chain of thought (CoT) reasoning using the Unsloth framework and parameter-efficient fine-tuning (PEFT) with QLoRA. Despite the fact that ROUGE-L scores remain stable at 0.3052 before and after fine-tuning, qualitative inspection revealed preservation of the reasoning style and improved inter-pretability without performance degradation. This demonstrates the feasibility of adapting compact LLMs for specialized reasoning tasks under constrained computational resources, such as Kaggle GPUs. The fine-tuned model and training pipeline are publicly released on the Hugging Face Hub, offering a reproducible baseline to support future research in interpretable and resource-efficient medical AI. Large Language Models (LLMs) such as GPT -4 [1], PaLM [2], and LLaMA-3.2 [3] have achieved state-of-the-art performance in a wide spectrum of natural language processing (NLP) tasks including summarization, machine translation, question answering, information retrieval, and dialogue generation.