Differentially Private Distributed Data Summarization under Covariate Shift

We envision Artificial Intelligence marketplaces to be platforms where consumers, with very less data for a target task, can obtain a relevant model by accessing many private data sources with vast number of data samples. One of the key challenges is to construct a training dataset that matches a target task without compromising on privacy of the data sources. To this end, we consider the following distributed data summarizataion problem. Given K private source datasets denoted by [D_i]_{i\in [K]} and a small target validation set D_v, which may involve a considerable covariate shift with respect to the sources, compute a summary dataset D_s\subseteq \bigcup_{i\in [K]} D_i such that its statistical distance from the validation dataset D_v is minimized. We use the popular Maximum Mean Discrepancy as the measure of statistical distance.