This paper focuses on generalization performance analysis for distributed algorithms in the framework of learning theory. Taking distributed kernel ridge regression (DKRR) for example, we succeed in deriving its optimal learning rates in expectation and providing theoretically optimal ranges of the number of local processors. Due to the gap between theory and experiments, we also deduce optimal learning rates for DKRR in probability to essentially reflect the generalization performance and limitations of DKRR. Furthermore, we propose a communication strategy to improve the learning performance of DKRR and demonstrate the power of communications in DKRR via both theoretical assessments and numerical experiments.

Abstract--This paper focuses on learning rate analysis of distributed kernel ridge regression for strong mixing sequences. Index Terms--Distributed learning, strong mixing sequences, kernel ridge regression, learning rate. I. Introduction With the development of data mining, data of massive size are collected in numerous application regions including Figure 1: Training flow of distributed learning recommendable systems, medical analysis, search engineering, financial analysis, online text, sensor network monitoring and social activity mining. A. Distributed learning cannot be reflected by data of small size [9], and creating new Distributed learning is a natural and preferable approach growth opportunities to combine and analyze industry data [3]. Finally, these distributively stored data.