Distributed Stochastic Multi-Task Learning with Graph Regularization

The goal of each machine is to find a good predictor for its own task, based on its own local data, as well as communicating with the other machines so as to leverage the similarity to other related tasks. Distributed multi-task learning lies between a homogeneous distributed learning setting (e.g. Shamir and Srebro, 2014), where all machines have data from the same source distribution, and inhomogeneous consensus problems (e.g. Ram et al., 2010; Boyd et al., 2011; Balcan et al., 2012), where each machine sees data from a different source, but the goal is to reach a single consensus predictor. In many distributed learning problems, different machines do indeed see different distributions.