VLLFL: A Vision-Language Model Based Lightweight Federated Learning Framework for Smart Agriculture

Abstract--In modern smart agriculture, object detection plays a crucial role by enabling automation, precision farming, and monitoring of resources. From identifying crop health and pest infestations to optimizing harvesting processes, accurate object detection enhances both productivity and sustainability. However, training object detection models often requires large-scale data collection and raises privacy concerns, particularly when sensitive agricultural data is distributed across farms. T o address these challenges, we propose VLLFL, a vision-language model-based lightweight federated learning framework (VLLFL). By training a compact prompt generator to boost the performance of the VLM deployed across different farms, VLLFL preserves privacy while reducing communication overhead. Experimental results demonstrate that VLLFL achieves 14.53% improvement in the performance of VLM while reducing 99.3% communication overhead. Spanning tasks from identifying a wide variety of fruits to detecting harmful animals in agriculture, the proposed framework offers an efficient, scalable, and privacy-preserving solution specifically tailored to agricultural applications. In recent years, smart agriculture has emerged as a transfor-mative approach to increase farming efficiency, reduce costs, and maintain environmental sustainability [1, 2]. By incorporating cutting-edge technologies such as the Internet of Things (IoT), artificial intelligence (AI), and advanced data analytics, smart agriculture offers improved monitoring and decision-making across the entire agricultural supply chain [3]. One crucial component of many smart farming solutions is object detection, which enables systems to identify crops, weeds, pests, and machinery in real-time through sensor networks or robotic platforms [4]. This real-time detection capability is instrumental in tasks like early pest detection [5] and precise harvesting [6], all of which can significantly enhance yield and utilization of resource [7]. Traditional object detection techniques, typically based on convolutional neural networks (CNNs) like Faster R-CNN [8] or YOLO [9], have demonstrated considerable success in classifying and localizing objects.[10].