Enhancing Domain Adaptation through Prompt Gradient Alignment

Deep learning has significantly advanced the field of computer vision, achieving remarkable performance in tasks such as image classification [1, 2, 3, 4, 5], object detection [6, 7, 8, 9], and semantic segmentation [10, 11, 12, 13]. However, the effectiveness of these deep learning models heavily relies on large amounts of labeled training data, which is often labor-intensive and expensive to collect. Moreover, the discrepancy between training data and real-world testing data can lead to substantial performance drops when models are deployed in practical settings [14, 15, 16]. To address these challenges, Unsupervised Domain Adaptation (UDA) has emerged as a pivotal solution. UDA aims to transfer knowledge from a labeled source domain to an unlabeled target domain in the presence of a domain shift, thereby enabling models to generalize well across different domains without requiring extensive labeled data for the target domain. This is often achieved by optimizing objective function on source domains and other auxiliary terms that encourage learning domain-invariant feature representations [17, 18, 19, 20] or enhance model robustness [21, 22, 23, 24], which mitigates the domain shift and improve the performance on unseen data. Nevertheless, aligning representations could potentially hurt the model performance due to the loss of discriminative features [25, 26]. Conceptually, our proposed method is orthogonal to these invariant feature learning methods, and they could complement each other.