Learning from Multiple Corrupted Sources, with Application to Learning from Label Proportions

We study the problem of binary classification in the setting where the learner does not have access to a conventional training data set with correctly labeled instances. In stead, the learner has access to several data sets for which the true labels have been randomly corrupted, with each data set having possibly different sample size and degree of corruption. Previous work has considere d learning from a single corrupted data set, but the problem considered here raises the natural question of how best to aggregate and weight the information from these multiple corrupted data sets according to t he sample size and degree of corruption. We extend the method of corruption corrected losses (Nataraja n et al., 2018) to this setting and establish a generalization error bound for kernel-based predictors. By optim izing this bound, we obtain a precise and interpretable scheme for aggregating the various corrupted sou rces according to the degree of corruption. We then apply our framework to the problem of learning from label pr oportions (LLP), which is another weak supervision setting for binary classification. In this problem, t raining data come in the form of bags. Each bag contains unlabeled feature vectors (patterns) and is an notated with the proportion of patterns arising from class 1. We argue that this problem can be reduced to th e first problem studied, and apply our results to obtain the most general theoretical analysis of this pro blem to date.