The supplemental material is organized as follows. Section A provides the results of all the additional synthetic experiments and real data results for various kernel-based methods and the detailed settings. Section B describes the algorithm details we use in Section A. In Section C, we provide some useful lemmas and all the technical proofs of the theoretical results in the main text. In this section, we provide more experiment results, including KRR (Section A.1), KQR for various Section A.7. A.1 Kernel ridge regression For the squared loss, we consider the following two examples. TIRW estimator still performs significantly better. A.2 Kernel quantile regression For the check loss, we consider the following two examples.

Such a phenomenon is illustrated in Figure 1 where the learned predictive function from the source data may significantly differ from the true function.

The supplemental material is organized as follows. Section A provides the results of all the additional synthetic experiments and real data results for various kernel-based methods and the detailed settings. Section B describes the algorithm details we use in Section A. In Section C, we provide some useful lemmas and all the technical proofs of the theoretical results in the main text. In this section, we provide more experiment results, including KRR (Section A.1), KQR for various Section A.7. A.1 Kernel ridge regression For the squared loss, we consider the following two examples. TIRW estimator still performs significantly better. A.2 Kernel quantile regression For the check loss, we consider the following two examples.

Covariate shift occurs prevalently in practice, where the input distributions of the source and target data are substantially different. Despite its practical importance in various learning problems, most of the existing methods only focus on some specific learning tasks and are not well validated theoretically and numerically. To tackle this problem, we propose a unified analysis of general nonparametric methods in a reproducing kernel Hilbert space (RKHS) under covariate shift. Our theoretical results are established for a general loss belonging to a rich loss function family, which includes many commonly used methods as special cases, such as mean regression, quantile regression, likelihood-based classification, and margin-based classification. Two types of covariate shift problems are the focus of this paper and the sharp convergence rates are established for a general loss function to provide a unified theoretical analysis, which concurs with the optimal results in literature where the squared loss is used. Extensive numerical studies on synthetic and real examples confirm our theoretical findings and further illustrate the effectiveness of our proposed method.