Our ideas also can be applied to the more general setting that the objective function only satisfies PL condition for one variable.

In the sequel, we empirically show the effect of different numbers of local updates on the fixed point. We consider cases withK = 1, K = 10, K = 20, K = 50. From Assumption 1, it is obvious thatgi(x,y) is convex-concave. Then, we conclude that there exists someη1 > 0 such that h(η) > 0, 0 < η < η1.

Byanalyzing Local SGDA under the ideal condition of no gradient noise, we show that generally it cannot guarantee exact convergence with constant stepsizes and thus suffers from slow rates of convergence.

To address this challenge, methods need to accommodate target domains with different temporal dynamics and be capable of doing so without seeing any target examples during pre-training.