Distributed Networked Multi-task Learning

--We consider a distributed multi-task learning scheme that accounts for multiple linear model estimation tasks with heterogeneous and/or correlated data streams. We assume that nodes can be partitioned into groups corresponding to different learning tasks and communicate according to a directed network topology. Each node estimates a linear model asynchronously and is subject to local (within-group) regularization and global (across groups) regularization terms targeting noise reduction and generalization performance improvement respectively. We provide a finite-time characterization of convergence of the estimators and task relation and illustrate the scheme's general applicability in two examples: random field temperature estimation and modeling student performance from different academic districts. Index T erms --Multi-task Learning, Distributed Optimization, Network-based computing systems, Multi-agent systems. N the current age of big data, many applications often face the challenge of processing large and complex datasets, which are usually not available in a single place but rather distributed across multiple locations. Approaches that require data to be aggregated in a central location may be subject to significant scalability and storage challenges. In other scenarios, data are scattered across different sites and owned by different individuals or organizations. Data privacy and security requirements make it difficult to merge such data in an easy way. In both contexts, Distributed Learning (DL) [1]-[3] can provide feasible solutions by building high-performance models shared among multiple nodes while maintaining user privacy and data confidentiality. DL aims to build a collective machine learning model based on the data from multiple computing nodes that can process and store data and are connected via networks. Nodes can utilize neighboring information to improve their own performance: rather than sharing raw data, they only exchange model information such as model parameters or gradients to avoid revealing sensitive information. This work was supported in part by the National Science Foundation under A ward ECCS-1933878 and in part by the Air Force Office of Scientific Research under Grant 15RT0767. Lingzhou Hong and Alfredo Garcia are with the Department of Industrial & Systems Engineering, Texas A&M University, College Station, TX 77843 USA (e-mail: { hlz, alfredo.garcia