Modern NLP systems increasingly rely on parameter-efficient adaptation methods to customize large pre-trained language models (PLMs) for new tasks without updating all model parameters. Among these, prompt tuning has become popular for its simplicity and low memory footprint: it learns a small set of continuous prompt embeddings while keeping the PLM frozen [5]. This allows adaptation to many tasks with minimal added parameters for both language models [2] and vision-language models [4]. Despite this efficiency, most existing prompt tuning methods are designed for single-task learning, where prompts are optimized independently for each task without sharing knowledge across them. This design is limiting in multi-task settings, where tasks often share semantic structure, labels, or data domains that can be exploited for transfer. Learning isolated prompts from scratch for each task misses opportunities for cross-task knowledge sharing--especially problematic in few-shot scenarios where data is scarce. Moreover, with only a single prompt vector per task, it becomes difficult to represent complex task relationships or adapt to varying label spaces, risking both underfitting of shared patterns and overfitting of task-specific noise. These challenges motivate the need for a more modular and transferable approach to prompt tuning that can share generalizable representations across tasks while preserving task-specific specialization in a parameter-efficient manner.

Remote sensing data is often distributed across multiple institutions, and due to privacy concerns and data-sharing restrictions, leveraging large-scale datasets in a centralized training framework is challenging. Federated learning offers a promising solution by enabling collaborative model training across distributed data sources without requiring data centralization. However, current Vision-Language Models (VLMs), which typically contain billions of parameters, pose significant communication challenges for traditional federated learning approaches based on model parameter updates, as they would incur substantial communication costs. In this paper, we propose FedRSCLIP, the first federated learning framework designed for remote sensing image classification based on a VLM, specifically CLIP. FedRSCLIP addresses the challenges of data heterogeneity and large-scale model transmission in federated environments by introducing Prompt Learning, which optimizes only a small set of tunable parameters. The framework introduces a dual-prompt mechanism, comprising Shared Prompts for global knowledge sharing and Private Prompts for client-specific adaptation. To maintain semantic coherence between shared and private prompts, we propose the Dual Prompt Alignment Constraint to balance global consistency and local adaptability across diverse client distributions. Additionally, to enhance cross-modal representation learning, we introduce the Cross-Modal Feature Alignment Constraint to align multimodal features between text and image prompts. To validate the effectiveness of our proposed model, we construct a Fed-RSIC dataset based on three existing remote sensing image classification datasets, specifically designed to simulate various federated learning configurations. Experimental results demonstrate the effectiveness and superiority of FedRSCLIP in remote sensing image classification.

In recent years, multi-task prompt tuning has garnered considerable attention for its inherent modularity and potential to enhance parameter-efficient transfer learning across diverse tasks. This paper aims to analyze and improve the performance of multiple tasks by facilitating the transfer of knowledge between their corresponding prompts in a multi-task setting. Our proposed approach decomposes the prompt for each target task into a combination of shared prompts (source prompts) and a task-specific prompt (private prompt). During training, the source prompts undergo fine-tuning and are integrated with the private prompt to drive the target prompt for each task. We present and compare multiple methods for combining source prompts to construct the target prompt, analyzing the roles of both source and private prompts within each method. We investigate their contributions to task performance and offer flexible, adjustable configurations based on these insights to optimize performance. Our empirical findings clearly showcase improvements in accuracy and robustness compared to the conventional practice of prompt tuning and related works. Notably, our results substantially outperform other methods in the field in few-shot settings, demonstrating superior performance in various tasks across GLUE benchmark, among other tasks. This achievement is attained with a significantly reduced amount of training data, making our method a promising one for few-shot settings.