A.1 Conjugate Derivations Cross-Entropy Loss: L(h,y) = cX

Pc i=1 yi = 1is satisfied, otherwise f (y) = by duality. A.2 Experiments on Binary Classification with Exponential Loss Here we present the results on a binary classification task over a synthetic dataset of 100 dimensional gaussian clusters. For Σ, similar to [23], we sample a diagonal matrix D, where each entry is sampled uniformly from a specified range, and a rotation matrix U from a HAAR distribution, giving Σ = UDUT. For the source data, we sample µ 1s,µ+1s,Σ 1s,Σ+1sas specified above with k = 0. Now to create a distribution shifted data of various severity, we sample µ 1t,µ+1t,Σ 1t,Σ+1tas specified above with k = 1, which are then used to sample the shifted data as follows: Exponential Loss for Binary Classification Let z be the classification score hθ(x). For logistic training loss, conjugate adaptation loss would default to entropy with sigmoid probability.